top of page
Implementation
PokerBot is divided into four main sections: the game model, the controller, the bot, and the user interface. The game model defines all of the objects and interactions used to play the game of poker, including pocket cards, wagers, and the current value of the pot. The bot uses reinforcement learning to develop a knowledge base of what actions maximize monetary reward. The controller takes input from either the bot or the human player and uses it to manipulate the game model. Finally, the user interface visually reflects changes in the model.
Model
The model defines every object and interaction in the poker game; it is simply a list of classes and methods that affect them.
​
The diagram below shows a simplified class structure of the poker endowment. It consists of one game class that contains players (which in turn possess a set of cards and personal funds), the deck, and the table cards. In this diagram, rectangular boxes represent classes and circles represent properties of the class. Each class has a variety of methods such as "deal_Card" and "bet", which are used to update the object's properties.
Bot
The component of artificial intelligence (AI) in this program is a bot that seeks to maximize money earned. As such, it "learns" to perform actions that give the greatest long term reward. In our simplified model of this process, a reward is only granted at the end of a poker game. If the bot loses, its is penalized by the amount of money it has wagered. If the bot wins, it is rewarded with all of the money on the table..
​
At its core, this functionality is a mapping from the state of the game environment to possible actions it can take, each of which has a learned reward value. Currently, our state value is a metric of the player's hand strength. The four actions a player may take at any time are: fold, check, call, and raise.
​
As the game is played, and state/action pairs are generated, the corresponding reward is updated in a nested dictionary. Note that this is a stochastic process: the first time the bot plays, it will take actions randomly. However, over time, it will converge to generate playing strategies. For more information on the implementation of the bot and the underlying AI theory, see The Bot.
Control
The controller takes input from the players and manipulates the model accordingly. That is, while the model gives a player the ability to fold, the controller is responsible for giving the command to fold. In addition to commanding player actions, the controller also dictates what state (preflop, flop, turn, river, showdown) the game is in. An important feature of the controller is that it does not care where it gets its input from. As a result, both the player and the bot can use the same commands to control the game.
View
The viewer takes the state of the model and represents it in a way that the user can understand. In the case of PokerBot, the viewer represents the model as a graphical interface on PyGame.
bottom of page