top of page
Starting Out
Putting the Bot in PokerBot
As we fleshed out our poker game, we also began to work on what would eventually become the poker bot. We envisioned a program that could learn by studying millions of hands from real-life poker tournaments. However, we quickly realized that this posed a major implementation issue. We wanted our bot to read both its cards and those of its opponent. However, none of the data sets we found online matched this vision. Instead, cards in the players hands could only be viewed after a showdown.
This posed a major problem for our bot-to-be. By looking at cards during the showdown, our bot would only get to study winning hands. More importantly, the cards of folded players are never shown. As a result it would be very difficult for our bot to learn when to fold and bluff, as the cards of the players who do these actions are rarely shown.
Starting Out
PokerBot began as a simple, standalone poker game that was playable from the command line. It was completely text based and did not include a semblance of artificial intelligence. We used this version to gain insight into the game of poker and how to incorporate our bot as a future player.
Poker Dataset where no player cards are shown
Building a User Interface
What is the point of a game if people cannot play it? From the start of the project, we knew that we wanted to have a graphical interface to facilitate human interaction. However, we were unsure about how that interface would be set up. Initially, we wanted the game to be playable over the Internet and thus moved towards Flask. Our first website was more of an image than a UI. It looked like a poker game but was not connected to any other programs. Its main purpose was to ensure Flask was working correctly.
Paying Our Debts
We took a divide and conquer approach to our project, splitting up the team and tackling the game, bot, and UI simultaneously. Unfortunately, we did not do a lot of planning beforehand and it showed when we tried to integrate the parts. Our greatest problem stemmed from inconsistent vocabulary
between the different programs. For instance, the term "Hand" was used to describe not only the round of the game, but also the physical cards in each player's hand, the cards on the table, and the cards scored during showdown! Cleaning up this code was essential for the continued development of our project.
In addition to integration, the poker code underwent a lot of organisational changes. These changes were designed to separate the model, view, and controller sections of the Python code into separate programs.
Rethinking the Bot
After much searching we were unable to find an appropriate data set for our bot to train off of. So we decided to pivot. Rather than have the bot learn from a data set, we decided to use reinforcement learning to train the bot. The bot would play against itself in our own game environment, such that we could be entirely transparent with all of the poker information it needs to train.
We decided to use a Monte Carlo method of making decisions based on Chen's poker hand strength formula. In this method, the bot builds a decision tree based around possible moves it can make with each hand. Moves that have had more success in the past are more likely to be played than poorly preforming moves. As the bot continues to play it learns by expanding these branches.
Putting It Together
After many days of code cleaning and bot development, it was time to integrate! Currently, we have succeeded in connecting the Monte Carlo Bot to the poker game, allowing it to play Bot vs. Human or Bot vs. Bot. However, we have not yet preformed any analyses on the bot to determine how effectively it is learning.
Future Work
In the near future, we intend to connect our poker game to the online user interface and to perform better analysis on the quality of our bot.
Command Line Poker
Sample Bot vs. Bot Game Flow
After a couple of weeks struggling with getting all of our desired functionalities into Flask, we eventually realized that PyGame would be better suited for our Model-View-Control schema.
bottom of page