This repository contains the codes for the paper:
Reinforcement learning based process optimization and strategy development in conventional tunneling
by Georg H. Erharter, Tom F. Hansen, Zhongqiang Liu and Thomas Marcher
published in Automation in Construction (Vol. 127; July 2021)
DOI: https://doi.org/10.1016/j.autcon.2021.103701
The paper was published as part of a collaboration on Machine Learning between the Institute of Rock Mechanics and Tunnelling (Graz University of Technology) and the Norwegian Geotechnical Institute (NGI) in Oslo.
Use the requirements.txt
file to download the required packages to run the code. We recommend using a package management system like conda for this purpose.
The code framework depends on a certain folder structure. The python files should be placed in the main directory. The set up should be done in the following way:
Reinforcement_Learning_for_Geotechnics
├── 02_plots
│ └── tmp
├── 04_checkpoints
├── 06_results
│ └── tmp
├── 00_main.py
├── 02_model_tester.py
├── 04_analyzer.py
├── A_utilities.py
├── B_generator.py
├── C_geotechnician.py
├── D_tunnel.py
└── E_plotter.py
Either set up the folder structure manually or on Linux run:
bash folder_structure.sh
00_main.py
... is the main executing file02_model_tester.py
... file that runs and tests individual checkpoints of already trained model for further analysis04_analyzer.py
... file that analyzes and visualizes the performance of agents tested with 02_model_tester.py
A_utilities.py
... is a library containing useful functions that do not directly belong to the environment or the agentB_generator.py
... part of the environment that generates a new geology for every episodeC_geotechnician.py
... part of the environment that evaluates the stability and also contains the RL agent itselfD_tunnel.py
... part of the environment that handles the rewards and updates the progress of the excavationE_plotter.py
... plotting functionalities to visualize the training progress or render episodes(inspired by Deeplizard)
Besides other references given in the paper, we especially want to highlight the Reinforcement Learning with Python
tutorial series of Sentdex which served as a basis for the agent in C_geotechnician.py
.