# Welcome to Catanatron! Catanatron is a high-performance simulator for The Settlers of Catan, designed to help you develop, test, and benchmark AI players. Whether you're a machine learning researcher, game developer, or just a Catan enthusiast curious about AI strategies, Catanatron provides the tools to explore the full strategic depth of the game at scale. ### What is Catanatron? Catanatron is an open-source project that allows you to: * Run thousands of Catan games per minute between different bots. * Develop your own AI players with simple Python interfaces. * Test strategies using weighted decision trees or custom algorithms. * Train reinforcement learning agents using an OpenAI Gym-compatible environment. * Web UI to watch, inspect, and play games against Catanatron! ### Getting Started You can start simple by playing against catanatron at [https://www.catanatron.com](https://www.catanatron.com/). Or jump right in:
Getting StartedRunning a simulation/files/R5Em0Bz07ds2ZW71NvJN/pages/CyH2xJQs9yWJ1S8BYNav
Simulation HooksLearn how to compute statistics on simulations/files/WI0Flywkj7Guvtj3PCrihttps://github.com/bcollazo/catanatron/blob/master/documentation/broken-reference/README.md
Graphical User InterfaceImprove the web UI/files/bbcQ1dZZbJTewNW28FTmhttps://github.com/bcollazo/catanatron/blob/master/documentation/broken-reference/README.md
# Installation ### Command Line Interface Installation 1. Clone the repository: ```bash git clone git@github.com:bcollazo/catanatron.git cd catanatron/ ``` 2. Create a virtual environment (requires Python 3.11 or higher) ```bash python -m venv venv source ./venv/bin/activate ``` 3. Install dependencies ```bash pip install -e . ``` 4. (Optional) Install developer and advanced dependencies ```bash pip install -e .[web,gym,dev] ``` ### Graphical User Interface Installation 1. Ensure you have Docker installed () 2. Run the `docker-compose.yaml` in the root folder of the repo: ```bash docker compose up ``` 3. Visit [http://localhost:3000](http://localhost:3000/) in your browser! # Quickstart

Catanatron's CLI is used to simulate thousands of games in the order of minutes

### Running your First Simulation After installing Catanatron, you should have access to the `catanatron-play` CLI. To run your first simulation run: ```bash catanatron-play --players=R,R,R,W --num=100 ``` For more information type: ```bash catanatron-play --help ``` ### Play Against Catanatron Go to [https://www.catanatron.com](https://www.catanatron.com/). Good luck! :raised\_hands: ### Running in Jupyter Notebook You can also do research with Catan in a Jupypter Notebook. To get started, try out our Overview\.ipynb: # Command Line Interface ### Basic Usage Examples * Run 1,000 games with Random Players ```bash catanatron-play --num 1000 --players R,R,R,R ``` * Run 10 1v1 games between VictoryPointPlayer and ValueFunctionPlayer ```bash catanatron-play --num 10 --players VP,F ``` * Pit 3-player Catan with basically no discard limit ```bash catanatron-play --num 1 --players W,F,AB:2 --config-discard-limit 999 ``` * Play 1v1 against ValueFunctionPlayer until 15 points with discard limit set to 9 (would not recommend :sweat\_smile:; its much better to play using the GUI) {% code overflow="wrap" %} ```bash catanatron-play --num 1 --players F,H --quiet --config-discard-limit 9 --config-vps-to-win 15 ``` {% endcode %}### Using Different Players Many of the Bots are ready to be used from the CLI using the `--players` flag. Specify players by using `:::...` syntax. For example, * `G:10` is GreedyPlayoutsPlayer with rollouts = 10 * `AB:3:True` is AlphaBetaPlayer with 3 ply look-ahead and prunning turned on * ... For more information, use: ```bash catanatron-play --help-players ```

catanatron-play --help-players output

### Saving Game Results You can save the output of games using the `--output` and `--output-format` flags. For example: #### JSON Format ```bash catanatron-play --num 5 --players F,F,R,R --output data/ --output-format json ``` Inspecting the `data` directory you should see all 5 games. ``` data/ ├── 2aba0ec4-51e7-47b6-b5a9-113874b2addf.json ├── 353a541b-59d2-4eb8-85ea-d6eb0d162e7f.json ├── 3c5f37ce-a73c-4180-91fd-28bf9f6dacd2.json ├── e31b00a3-845c-4bfd-92bd-c4a42cbafa27.json └── ee547837-d5e2-4d9b-8420-583b2fdd793d.json 1 directory, 5 files ``` #### Other Formats You can also generate data in **CSV** and **Parquet** for Machine Learning and similar applications. See more at [Data and Machine Learning](/advanced/data-and-machine-learning). # Python Library You can also use `catanatron` package directly which provides a core implementation of the Settlers of Catan game logic. ```python from catanatron import Game, RandomPlayer, Color # Play a simple 4v4 game players = [ RandomPlayer(Color.RED), RandomPlayer(Color.BLUE), RandomPlayer(Color.WHITE), RandomPlayer(Color.ORANGE), ] game = Game(players) print(game.play()) # returns winning color ``` ### Iterating Over Plys (ticks) Instead of using `game.play()` to play a game until completion, you can iterate step-by-step on each ply (decision point) like so: ```python game = Game(players) while game.winning_color() is None: print(game.state) action = game.play_tick() print(action) ``` ### Debugging Game State Inspecting the game state mid-simulation in the example above can be challenging. Even though inspecting the following is helpful: ```python print(game.state.board.map.tiles) print(game.state.board.buildings) print(game.state.board.roads) print(game.state.player_state) ``` Its often best to see them in the GUI. For this, if you have the GUI Docker Services running alongside this process you can open the game at this state in the GUI with the `open_link` function: ```python from catanatron.web.utils import open_link open_link(game) # opens game in browser ``` # Simulation Hooks The `Accumulator` class allows you to hook into important events during simulations. For example, write a file like `mycode.py` and have: ```python from catanatron import ActionType from catanatron.cli import SimulationAccumulator, register_cli_accumulator class PortTradeCounter(SimulationAccumulator): def before_all(self): self.num_trades = 0 def step(self, game_before_action, action): if action.action_type == ActionType.MARITIME_TRADE: self.num_trades += 1 def after_all(self): print(f'There were {self.num_trades} trades with the bank!') register_cli_accumulator(PortTradeCounter) ``` Then `catanatron-play --code=mycode.py` will count the number of trades in all simulations. # Graphical User Interface We provide a [docker-compose.yml](https://github.com/bcollazo/catanatron/blob/master/docker-compose.yml) with everything needed to play and watch games (useful for debugging). It contains all the web-server infrastructure needed to render a game in a browser.

Catanatron Web UI

To use, ensure you have [Docker Compose](https://docs.docker.com/compose/install/) installed, and run (from this repo's root): ```bash docker compose up ``` You should now be able to visit [http://localhost:3000](http://localhost:3000/) and play! You can also (in a new terminal window) install the `[web]` subpackage and use the `--db` flag to make the catanatron-play simulator save the game in the database for inspection via the web server. ```bash pip install .[web] catanatron-play --players=W,W,W,W --db --num=1 ``` The link should be printed in the console. {% hint style="info" %} A great contribution would be to make the Web UI allow to step forwards and backwards in a game to inspect it (ala chess.com)! {% endhint %} # Examples WIP (see examples/ folder in repo) # Making Catanatron Stronger ### AI Leaderboard Catanatron will always refer to the best bot in this leaderboard. The best bot right now is `AlphaBetaPlayer` with n = 2. Here a list of bots strength. Experiments done by running 1000 (when possible) 1v1 games against previous in list. | Player | % of wins in 1v1 games | num games used for result | | -------------------- | ------------------------------------------- | ------------------------- | | AlphaBeta(n=2) | 80% vs ValueFunction | 25 | | ValueFunction | 90% vs GreedyPlayouts(n=25) | 25 | | GreedyPlayouts(n=25) | 100% vs MCTS(n=100) | 25 | | MCTS(n=100) | 60% vs WeightedRandom | 15 | | WeightedRandom |

60% vs Random
50% vs VictoryPoint

| 1000 | | VictoryPoint | 60% vs Random | 1000 | | Random | - | - | ### Making Catanatron Bot Stronger The best bot right now is Alpha Beta Search with a hand-crafted value function. One of the most promising ways of improving Catanatron is to have your custom player inhert from ([`AlphaBetaPlayer`](https://github.com/bcollazo/catanatron/blob/master/catanatron/catanatron/players/minimax.py)) and set a better set of weights for the value function. You can also edit the value function and come up with your own innovative features! For more sophisticated approaches, see example player implementations in [catanatron/catanatron/players](https://github.com/bcollazo/catanatron/blob/master/catanatron/catanatron/players/README.md) If you find a bot that consistently beats the best bot right now, please submit a Pull Request! :) # Creating Custom Bots Implement your own bots by creating a file (e.g. `myplayers.py`) with some `Player` implementations, and registering it for CLI usage: ```python from catanatron import Player from catanatron.cli import register_cli_player class FooPlayer(Player): def decide(self, game, playable_actions): """Should return one of the playable_actions. Args: game (Game): complete game state. read-only. playable_actions (Iterable[Action]): options to choose from Return: action (Action): Chosen element of playable_actions """ # ===== YOUR CODE HERE ===== # As an example we simply return the first action: return playable_actions[0] # type: ignore # ===== END YOUR CODE ===== register_cli_player("FOO", FooPlayer) ``` Run it by passing the source code to `catanatron-play`: ```bash catanatron-play --code=myplayers.py --players=R,R,R,FOO --num=10 ``` # Gymnasium Interface For reinforcement learning purposes, we provide an Open AI Gym / Gymnasium environment. To use, in the root of the catanatron repository: ```bash pip install -e .[gym] ``` Make your training loop, ensuring to respect `info['valid_actions']` : ```python import random import gymnasium import catanatron.gym env = gymnasium.make("catanatron/Catanatron-v0") observation, info = env.reset() for _ in range(1000): # your agent here (this takes random actions) action = random.choice(info["valid_actions"]) observation, reward, terminated, truncated, info = env.step(action) done = terminated or truncated if done: observation, info = env.reset() env.close() ``` For `action` documentation see [here](https://catanatron.readthedocs.io/en/latest/catanatron.gym.envs.html#catanatron.gym.envs.catanatron.gym.CatanatronEnv.action_space). For `observation` documentation see [here](https://catanatron.readthedocs.io/en/latest/catanatron.gym.envs.html#catanatron.gym.envs.catanatron.gym.CatanatronEnv.observation_space). You can access `env.unwrapped.game.state` and build your own "observation" (features) vector as well. For evaluation and using your model in the simulator for testing / benchmarking you might want to checkout: ### Stable-Baselines3 Example Catanatron works well with SB3, and better with the Maskable models of the [SB3 Contrib](https://stable-baselines3.readthedocs.io/en/master/guide/sb3_contrib.html) repo. Here a small example of how it may work. ```python import gymnasium import numpy as np from sb3_contrib.common.maskable.policies import MaskableActorCriticPolicy from sb3_contrib.common.wrappers import ActionMasker from sb3_contrib.ppo_mask import MaskablePPO import catanatron.gym def mask_fn(env) -> np.ndarray: valid_actions = env.unwrapped.get_valid_actions() mask = np.zeros(env.action_space.n, dtype=np.float32) mask[valid_actions] = 1 return np.array([bool(i) for i in mask]) # Init Environment and Model env = gymnasium.make("catanatron/Catanatron-v0") env = ActionMasker(env, mask_fn) # Wrap to enable masking model = MaskablePPO(MaskableActorCriticPolicy, env, verbose=1) # Train model.learn(total_timesteps=10_000) ``` ### Configuration You can also configure what map to use, how many vps to win, among other variables in the environment, with the `config` keyword argument. See source for details. ```python import gymnasium from catanatron import Color from catanatron.players.weighted_random import WeightedRandomPlayer import catanatron.gym def my_reward_function(game, p0_color): winning_color = game.winning_color() if p0_color == winning_color: return 100 elif winning_color is None: return 0 else: return -100 # 3-player catan on a "Mini" map (7 tiles) until 6 points. env = gymnasium.make( "catanatron.gym/Catanatron-v0", config={ "map_type": "MINI", "vps_to_win": 6, "enemies": [ WeightedRandomPlayer(Color.RED), WeightedRandomPlayer(Color.ORANGE), ], "reward_function": my_reward_function, "representation": "mixed", }, ) ``` # Data and Machine Learning ## Generating Data Using the CLI you can generate data, suitable for Machine Learning and Data Analysis. #### CSV With `--output-format csv` you can generate the CSVs like so: ```bash catanatron-play --num 5 --players F,F,R,R --output data/ --output-format csv ``` This generates 4 GZIP CSVs: * **samples.csv.gz:** One row per game state at each [ply](/core-concepts). * **actions.csv.gz:** Integers representing actions taken by each player at each ply. * **rewards.csv.gz:** Some common returns and rewards with which to label how effective actions taken by bots where. See Reinforcement Learning section on [Gymnasium Interface](/advanced/openapi) * **main.csv.gz:** Simply a concatenation of the above 3 CSVs Using `--output-format csv` will continuously simply append to these 4 files more and more plys even if from different games. #### Parquet [Parquet format](https://parquet.apache.org/) is also supported. One file per game is generated (like in JSON format) ```bash catanatron-play --num 5 --players F,R --output data/ --output-format parquet ``` ## Board Tensors Representing the Catan Board State for Machine Learning purposes is challenging using tabular data. The best we can often do is have many features like `TILE11_IS_BRICK` or `TILE3_PROBA` to represent what resource and what number lie in what tile. Similarly columns like `NODE9_P2_CITY` and so on to represent if Player 2 has a City in Node 9. To aid in this regard, you can also generate a 3D Tensor for each game state capturing the spatial relationship of these features. You can generate it when using `--output-format parquet` or `--output-format csv` by using the `--include-board-tensors` flag. For example: ```bash catanatron-play --num 5 --players AB:2,AB:2 --output data/ --output-format csv --include-board-tensor ``` Its a dedicated flag since it makes simulations a bit slower. ### Dimensions and Channel Descriptions One sample has shape `(WIDTH=21, HEIGHT=11, CHANNELS=2*N+12)` where CHANNELS depend on the number of players of the game `N`. #### Player Building Channels (Indexes: 0 to 2N - 1) If a Player 0 (the player with the perspective from which we take the sample) has a settlement in the 8 ORE - 5 SHEEP - 4 BRICK node below (the one marked as \[6, 8]), then that means `board_tensor[6, 8, 0]` would be `1`. If it was a city, it would be `2` If Player 3 has a road between the 10 WOOD and 11 WHEAT, the coordinate `board_tensor[10, 3, 2]` would be `1` (2 meaning that it is player 3). The vertical "edges" are captured in the odd rows of the tensor.
#### Tile Channels (Indexes: 2N to 2N + 4) If there are `N` players, channels `2N` to `2N + 5` talk about the resource yield of each node. The following is an index map of the nodes. So for example... `board_tensor[6, 2, 8]` in this board for a 4 player game should be `0.08333333333333333` since the probability of rolling a 10 is \~8.33% and this is the first resource plane (which corresponds to Wood). It is also the case for `board_tensor[8, 2, 8]` and `board_tensor[10, 2, 8]` . But for `board_tensor[6, 4, 8]` it should be the probability of rolling a 10 or a 3 (\~5.55%), so it should give `0.1388888888888889`. #### Robber Plane (Index: 2N + 5) There is a robber plane that places a `1` on all nodes of the tile it is blocking. As shown below for example (this is the desert above).
#### Port Planes (Indexes: 2N + 6 to 2N + 11) There are 6 more planes with `1` if the node enables that trading rate. The 3:1 is the very last channel. So for example, for the above board, `board_tensor[6, 10, 19]` and `board_tensor[4, 10, 19]` are `1`, capturing the 3:1 port of the 4 BRICK tile. # Contributing Any and all contributions are more than welcome! ## Running Tests To develop for Catanatron, install the development dependencies and use the following test suite: ```bash pip install ".[web,gym,dev]" coverage run --source=catanatron -m pytest tests/ && coverage report ``` Or you can run the suite in watch-mode with: ```bash ptw --ignore=tests/integration_tests/ --nobeep ``` ## Architecture The code is divided in three main components (folders): * **catanatron**: The pure python implementation of the game logic. Uses `networkx` for fast graph operations. It is pip-installable (see [pyproject.toml](https://github.com/bcollazo/catanatron/blob/master/pyproject.toml)) and can be used as a Python package. The implementation of this follows the idea of Game Trees (see ) so that it lends itself for Tree-Searching Bots and Reinforcement Learning Environment Loops. Every "ply" is advanced with the `.play_tick` function. See more on Code Documentation site: * **catanatron.web**: An extension package (optionally installed) that contains a Flask web server in order to serve game states from a database to a Web UI. The idea of using a database, is to ease watching games played in a different process. It defaults to using an ephemeral in-memory sqlite database. Also pip-installable with `pip install catanatron[web]`. * **catanatron.gym**: Gymnasium interface to Catan. Includes a configurable 1v1 environment and a vector-friendly representations of states and actions. This can be pip-installed independently with `pip install catanatron[gym]`, for more information see [catanatron/gym/README.md](https://github.com/bcollazo/catanatron/blob/master/catanatron/catanatron/gym/README.md). * **catanatron.cli**: A rich-powered CLI that enables the `catanatron-play` console script. Can be used to play games in bulk, create machine learning datasets of games, and more! * **catantron\_experimental**: A collection of unorganized scripts with contain many failed attempts at finding the best possible bot. Its ok to break these scripts. Its pip-installable. * **ui**: A React web UI to render games. This is helpful for debugging the core implementation. We decided to use the browser as a randering engine (as opposed to the terminal or a desktop GUI) because of HTML/CSS's ubiquitousness and the ability to use modern animation libraries in the future ( or ). ## Running Components Individually As an alternative to running the project with Docker, you can run the web client and server in two separate tabs. ### React App ```bash cd ui/ npm install npm start ``` This can also be run via Docker independently (after building): ```bash docker build -t bcollazo/catanatron-react-ui:latest ui/ docker run -it -p 3000:3000 bcollazo/catanatron-react-ui ``` ### Flask Web Server Ensure you are inside a virtual environment with all dependencies installed and use `flask run`. This will use SQLite by default. ```bash pip install -e .[web] FLASK_DEBUG=1 FLASK_APP=catanatron.web/catanatron.web flask run ``` This can also be run via Docker independently (after building): ```bash docker build -t bcollazo/catanatron-server:latest . -f Dockerfile.web docker run -it -p 5001:5001 bcollazo/catanatron-server ``` ## Useful Commands These are other potentially useful commands while developing catanatron #### TensorBoard For watching training progress, use `keras.callbacks.TensorBoard` and open TensorBoard: ```bash tensorboard --logdir logs ``` #### Docker GPU TensorFlow ```bash docker run -it tensorflow/tensorflow:latest-gpu-jupyter bash docker run -it --rm -v $(realpath ./notebooks):/tf/notebooks -p 8888:8888 tensorflow/tensorflow:latest-gpu-jupyter ``` #### Testing Performance ```bash pyinstrument -r html --from-path catanatron-play --players AB:2,AB:2 ``` ```bash python -m cProfile -o profile.pstats examples/play_batch_example.py snakeviz profile.pstats ``` ```bash pytest --benchmark-compare=0001 --benchmark-compare-fail=mean:10% --benchmark-columns=min,max,mean,stddev ``` #### Head Large Datasets with Pandas ```python import pandas as pd x = pd.read_csv("data/mcts-playouts-labeling-2/labels.csv.gzip", compression="gzip", iterator=True) x.get_chunk(10) ``` Building Sphinx Code Documentation Site ```bash pip install -r docs/requirements.txt sphinx-quickstart docs sphinx-apidoc -o docs/source catanatron sphinx-build -b html docs/source/ docs/build/html ``` #### Publishing to PyPi (Outdated) catanatron Package ```bash make build PACKAGE=catanatron make upload PACKAGE=catanatron make upload-production PACKAGE=catanatron ``` catanatron\_gym Package ```bash make build PACKAGE=catanatron_gym make upload PACKAGE=catanatron_gym make upload-production PACKAGE=catanatron_gym ``` ## Ideas for Contribution * Improve `catanatron` package running time performance. * Continue refactoring the State to be more and more like a primitive `dict` or `array`. (Copies are much faster if State is just a native python object). * Move RESOURCE to be ints. Python `enums` turned out to be slow for hashing and using. * Move the `.action_records` action log concept to the Game class. This way MCTS algorithms that just need copy games for the purposes of rollouts, don't need to pay for copying the action\_records, but AlphaBeta players can still use the log for undoing actions. * Remove `.current_prompt`. It seems its redundant with (is\_moving\_knight, etc...) and not needed. * Improve AlphaBetaPlayer * Explore and improve prunning * Use Bayesian Methods or [SPSA](https://www.chessprogramming.org/SPSA) to tune weights and find better ones. * Research! * Deep Q-Learning * Simple Alpha Go * Try Tensorforce with simple action space. * Try simple flat CSV approach but with AlphaBeta-generated games. * Features * Continue implementing actions from the UI (not all implemented). * A Terminal UI? (for ease of debugging) # Core Concepts # Game Trees ### Plys and Turns ### Nodes / State ### Edges / Action