This is a multi-agent version of TORCS, for multi-agent reinforcement learning. In other words, the multiple cars running simultaneously on a track can be controlled by different control algorithms - heuristic, reinforcement learning-based, etc.
Please check out the updated version of MADRaS here!
It is assumed that you have TORCS installed (tested on version 1.3.6) from the source code on a machine with Ubuntu 14.04/16.04 LTS.
Install the scr-client as follows:
cd scr-patch
sh do_patch.sh
(do_unpatch.sh
to revert the modifications) cd ../
./configure
make -j4
sudo make install -j4
sudo make datainstall -j4
10 scr_server car should be available in the race configurations now.
scr-client-cpp.tgz
in your base TORCS directory.scr-client-cpp
.cd scr-client-cpp
make -j4
./client
./client port:3002
Typical values are between 3001 and 3010 (3001 is the default)
scr_*
)torcs
python3 playGame.py 3001
python3 playGame.py 3002
In the game loop in playGame.py
, the action at every timestep a_t
can be supplied by any algorithm.
Note :
playGame_DDPG.py
has the code for a sample RL agent learning with the DDPG algorithm, while playGame.py
has a dummy agent which just moves straight at every timestep.scr
car and as many as traffic cars as you want (preferably chenyi*
1, since they're programmed to follow individual lanes at speeds low enough for the agent to learn to overtake)python3 playGame_DDPG.py 3001
Sample results for a DDPG agent learned to drive in traffic are available here.
Do check out the wiki for this project for in-depth information about TORCS and getting Deep (Reinforcement) Learning to work on it.
1 The chenyi*
cars can be installed from Princeton's DeepDrive project, which also adds a few maps from training and testing the agents. The default cars in TORCS are all programmed heuristic racing agents, which do not serve as good stand-ins for 'traffic'. Hence, using chenyi's code is highly recommended.
The multi-agent learning simulator was developed by Abhishek Naik, extending ugo-nama-kun
's gym-torcs
, and yanpanlau
's project under the guidance of Anirban Santara, Balaraman Ravindran, and Bharat Kaul, at Intel Labs.
We believe MADRaS will enable new and veteran researchers in academia and the industry to make the dream of fully autonomous driving a reality. Towards the same, we believe that unlike the closed-source secretive technologies of the big players, this project will enable the community to work towards this goal together, pooling in thoughts and resources to achieve this dream faster. Hence, we're highly appreciative of all sorts of contributions, big or small, from fellow researchers and users :