search

LucasAlegre / sumo-rl

Reinforcement Learning environments for Traffic Signal Control with SUMO. Compatible with Gymnasium, PettingZoo, and popular RL libraries.
https://lucasalegre.github.io/sumo-rl
MIT License
734 stars 198 forks source link

Deploy independent model on the network #202

Open Kunal-Kumar-Sahoo opened 5 months ago

Kunal-Kumar-Sahoo commented 5 months ago

Given a traffic network with multiple intersections, I want to deploy a single DQN to all the individual traffic light signal. How can I do that?

LucasAlegre commented 5 months ago

SUMO-RL provides environments with the PettingZoo API. You can use any RL library that supports it. See https://pettingzoo.farama.org/tutorials/sb3/ for examples of how to use different libraries with PettingZoo environments, both for independent and centralized agents.

Kunal-Kumar-Sahoo commented 5 months ago

While implementing the example sumo_rl/experiments/sb3_grid4x4.py I get the following error:

Using libsumo as traci as requested by environment variable.
/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/pettingzoo/utils/conversions.py:132: UserWarning: The base environment `sumo_rl_v0` does not have a `render_mode` defined.
  warnings.warn(
Traceback (most recent call last):
  File "/home/kunalkumarsahoo/Playground/CiSTUP/research_internship/multi_agent/agent.py", line 22, in <module>
    env = ss.pettingzoo_env_to_vec_env_v1(env)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/supersuit/vector/vector_constructors.py", line 89, in pettingzoo_env_to_vec_env_v1
    return MarkovVectorEnv(parallel_env)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/supersuit/vector/markov_vector_wrapper.py", line 20, in __init__
    self.render_mode = par_env.unwrapped.render_mode
AttributeError: 'SumoEnvironmentPZ' object has no attribute 'render_mode'

Upon checking these files where the error has occurred I could not find SumoEnvironmentPZ class. How can I resolve this issue?

LucasAlegre commented 5 months ago

Thanks for spotting this. I fixed in the last commit: https://github.com/LucasAlegre/sumo-rl/commit/2a8b6e5400707debe9faead2291477285f5dfdb9

Kunal-Kumar-Sahoo commented 5 months ago

Now I get this error:

Traceback (most recent call last):
  File "/home/kunalkumarsahoo/Playground/CiSTUP/research_internship/multi_agent/agent.py", line 50, in <module>
    model.learn(total_timesteps=50000)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/stable_baselines3/ppo/ppo.py", line 315, in learn
    return super().learn(
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 287, in learn
    total_timesteps, callback = self._setup_learn(
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/stable_baselines3/common/base_class.py", line 423, in _setup_learn
    self._last_obs = self.env.reset()  # type: ignore[assignment]
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/stable_baselines3/common/vec_env/vec_monitor.py", line 70, in reset
    obs = self.venv.reset()
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/supersuit/vector/sb3_vector_wrapper.py", line 22, in reset
    observations, self.reset_infos = self.venv.reset()
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/supersuit/vector/concat_vec_env.py", line 45, in reset
    _obs, _info = self.vec_envs[i].reset(options=options)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/supersuit/vector/markov_vector_wrapper.py", line 57, in reset
    _observations, infos = self.par_env.reset(seed=seed, options=options)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/pettingzoo/utils/conversions.py", line 179, in reset
    self.aec_env.reset(seed=seed, options=options)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/pettingzoo/utils/wrappers/order_enforcing.py", line 118, in reset
    super().reset(seed=seed, options=options)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/pettingzoo/utils/wrappers/base.py", line 38, in reset
    self.env.reset(seed=seed, options=options)
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/pettingzoo/utils/wrappers/base.py", line 38, in reset
    self.env.reset(seed=seed, options=options)
  File "/home/kunalkumarsahoo/Playground/CiSTUP/sumo-rl/sumo_rl/environment/env.py", line 542, in reset
    self.env.reset(seed=seed, options=options)
  File "/home/kunalkumarsahoo/Playground/CiSTUP/sumo-rl/sumo_rl/environment/env.py", line 255, in reset
    self._start_simulation()
  File "/home/kunalkumarsahoo/Playground/CiSTUP/sumo-rl/sumo_rl/environment/env.py", line 241, in _start_simulation
    self.sumo.gui.setSchema(traci.gui.DEFAULT_VIEW, "real world")
  File "/home/kunalkumarsahoo/mambaforge/envs/cistup/lib/python3.10/site-packages/libsumo/libsumo.py", line 2060, in setSchema
    return _libsumo.gui_setSchema(viewID, schemeName)
libsumo.libsumo.TraCIException: View 'View #0' is not known
LucasAlegre commented 5 months ago

This seems to be a problem with the GUI. Are you running in a cluster or something without a screen monitor? Try running without the GUI rendering.