search

flow-project / flow

Computational framework for reinforcement learning in traffic control
MIT License
1.06k stars 374 forks source link

Problem replaying policy on sumo GUI using visualizer #920

Open habibi1195 opened 4 years ago

habibi1195 commented 4 years ago

Bug Description

When using the visualizer_rllib.py, there is an error thrown by rllib.py in the function get_flow_params, when the net_module is being defined. The error occurs because network is defined as flow_params['network'], which is a dict.

Bug Reproduce

File in which the bug occurred: visualizer_rllib.py -> 'rllib.py'

The exact command that you did run that caused the bug: python visualizer_rllib.py ~/ray_results/experiment_directory checkpoint_number

Output Log

Full error output log:

E0504 16:31:13.072916200   12659 socket_utils_common_posix.cc:208] check for SO_REUSEPORT: {"created":"@1588624273.072875400","description":"Protocol not available","errno":92,"file":"external/com_github_grpc_grpc/src/core/lib/iomgr/socket_utils_common_posix.cc","file_line":185,"os_error":"Protocol not available","syscall":"getsockopt(SO_REUSEPORT)"}
E0504 16:31:13.073060000   12659 socket_utils_common_posix.cc:313] setsockopt(TCP_USER_TIMEOUT) Protocol not available
Traceback (most recent call last):
  File "visualizer_rllib.py", line 386, in <module>
    visualizer_rllib(args)
  File "visualizer_rllib.py", line 70, in visualizer_rllib
    flow_params = get_flow_params(config)
  File "/home/ha55/Desktop/flow/flow/flow/utils/rllib.py", line 180, in get_flow_params
    net_module = __import__(net_loc, fromlist=[network])
  File "<frozen importlib._bootstrap>", line 1014, in _handle_fromlist
TypeError: Item in ``from list'' must be str, not dict
habibi1195 commented 4 years ago

Note that, when testing the visualizer via the tutorial page, it also throws an error. The command I use (from the flow/tutorials folder)

python ../flow/visualize/visualizer_rllib.py data/trained_ring 200 --horizon 2000

The error output is as follows:

2020-05-04 16:33:31,436 INFO services.py:1123 -- View the Ray dashboard at localhost:8266
E0504 16:33:31.634057900   12955 socket_utils_common_posix.cc:208] check for SO_REUSEPORT: {"created":"@1588624411.634004600","description":"Protocol not available","errno":92,"file":"external/com_github_grpc_grpc/src/core/lib/iomgr/socket_utils_common_posix.cc","file_line":185,"os_error":"Protocol not available","syscall":"getsockopt(SO_REUSEPORT)"}
E0504 16:33:31.634199100   12955 socket_utils_common_posix.cc:313] setsockopt(TCP_USER_TIMEOUT) Protocol not available
2020-05-04 16:33:31,771 INFO trainer.py:428 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2020-05-04 16:33:31,917 WARNING deprecation.py:30 -- DeprecationWarning: `sample_batch_size` has been deprecated. Use `rollout_fragment_length` instead. This will raise an error in the future!
Traceback (most recent call last):
  File "../flow/visualize/visualizer_rllib.py", line 386, in <module>
    visualizer_rllib(args)
  File "../flow/visualize/visualizer_rllib.py", line 152, in visualizer_rllib
    agent = agent_cls(env=env_name, config=config)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 93, in __init__
    Trainer.__init__(self, config, env, logger_creator)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 455, in __init__
    super().__init__(config, logger_creator)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trainable.py", line 174, in __init__
    self._setup(copy.deepcopy(self.config))
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 596, in _setup
    self._init(self.config, self.env_creator)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 121, in _init
    self.config["num_workers"])
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 666, in _make_workers
    logdir=self.logdir)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 61, in __init__
    RolloutWorker, env_creator, policy, 0, self._local_config)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 271, in _make_worker
    _fake_sampler=config.get("_fake_sampler", False))
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 287, in __init__
    self.env = _validate_env(env_creator(env_context))
  File "/home/ha55/Desktop/flow/flow/flow/utils/registry.py", line 132, in create_env
    return gym.envs.make(env_name)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/gym/envs/registration.py", line 156, in make
    return registry.make(id, **kwargs)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/gym/envs/registration.py", line 101, in make
    env = spec.make(**kwargs)
  File "/home/ha55/anaconda3/envs/flow/lib/python3.6/site-packages/gym/envs/registration.py", line 73, in make
    env = cls(**_kwargs)
  File "/home/ha55/Desktop/flow/flow/flow/envs/ring/wave_attenuation.py", line 87, in __init__
    super().__init__(env_params, sim_params, network, simulator)
  File "/home/ha55/Desktop/flow/flow/flow/envs/base.py", line 130, in __init__
    self.net_params = self.network.net_params
AttributeError: type object 'RingNetwork' has no attribute 'net_params'
(pid=raylet) E0504 16:33:31.603704100   12976 socket_utils_common_posix.cc:208] check for SO_REUSEPORT: {"created":"@1588624411.603686700","description":"Protocol not available","errno":92,"file":"external/com_github_grpc_grpc/src/core/lib/iomgr/socket_utils_common_posix.cc","file_line":185,"os_error":"Protocol not available","syscall":"getsockopt(SO_REUSEPORT)"}
(pid=raylet) E0504 16:33:31.603922500   12976 socket_utils_common_posix.cc:313] setsockopt(TCP_USER_TIMEOUT) Protocol not available
(pid=raylet) E0504 16:33:31.610868400   12976 socket_utils_common_posix.cc:313] setsockopt(TCP_USER_TIMEOUT) Protocol not available

In fact, when I use my own experiment and rewrite rllib.py to directly obtain the correct module and class of the network, it reproduces the above error identical to that from the ring experiment above.