AchiaIR / solve_sokoban

Solve Sokoban game using Deep Reinforcement Learning
5 stars 0 forks source link

Solve Sokoban with Deep Reinforcement Learning

πŸ”΄ Table of Contents

─────────────────────────────────────────────

πŸ”΄ About The Project

This project is a showcase of how the Sokoban game, a difficult transportation puzzle where boxes are moved to storage locations, can be solved using Reinforcement Learning approach. Here we use Deep Q-Network, A model-free algorithm, Q-Network learns by trial and error. It gauges the value of an action in a particular state and refines its strategy over time, making it adaptable and dynamic. Deep Q-Network means making use of deep neural network as "Q-table", the way the algorithm takes decisions.

─────────────────────────────────────────────

πŸ”΄ Overview

Explain about the 2 parts of the project - the Sokoban game, and the Deep Q-Network algorithm:

─────────────────────────────────────────────

πŸ”΄ Project Content Description

Python Content Description

Other supporting Content Description

─────────────────────────────────────────────

πŸ”΄ Getting Started

Prerequisites

Installation

  1. Clone this repository or download and extract the ZIP file:

    git clone https://github.com/AchiaIR/solve_sokoban.git

  2. Navigate to the directory where you cloned or extracted the project:

    cd solve_sokoban

  3. Install the necessary dependencies:

    pip install -r requirements.txt

    or:

    pip install .

    or:

    python setup.py install

  1. Clone this repository or download and extract the ZIP file:

    git clone https://github.com/AchiaIR/solve_sokoban.git

  2. Navigate to the directory where you cloned or extracted the project:

    cd solve_sokoban

  3. Install the necessary dependencies:

    pip3 install -r requirements.txt

    or:

    pip3 install .

    or:

    pip3 install setuptools

    python3 setup.py install

Usage

Run the run.bat script:

run

you can play with the number of boxes:

run 2

  1. Make the script executable (only need to do this once):

    chmod +x ./run.sh

  2. Run the file.sh script:

    ./run.sh

    you can play with the number of boxes:

    run 2

─────────────────────────────────────────────

πŸ”΄ Advanced Usage

Since the algorithm is based on training a deep neural netowrk, running it with a random environement, where it creates a new game structure each time, means it will have to train it from scratch in each run. This may take a long time on CPU. Therefor I trained networks for a constant seed (seed=2), and upload the pth files here, so when one runs it it will create the same game structure and will solve it using the pre-trained networks.

If you wish to train it yourself and explore it further, you can uncomment the function SetAlgorithmWithTraining in solve_sokoban.py, and also the call to that finction in main function, and run it with traininig from scratch. You better have a GPU installed f you choose to do so.

You also may play with the paramteres, explor the best parameters for fast convergence, and maybe chnaging the reward function and see how it affects the solving abilities. Another aspect to test is the number of episodes and maximum steps in each episode - the less the better, on a condition the convergence percentage (how many times you run it succesfully on a new structure) is high enough.

I also add here the google colab notebook for an easy exploration.

─────────────────────────────────────────────

πŸ”΄ Acknowledgments

Based on a project in Reinforcement Learning course, Reichman University