search

flow-project / flow

Computational framework for reinforcement learning in traffic control
MIT License
1.05k stars 370 forks source link

Exceptions in Traffic Light Grid RL Examples #787

Open acjoseph opened 4 years ago

acjoseph commented 4 years ago

Bug Description

I am running into an issue with the traffic_light_grid examples from both the stable_baselines and rllib set of examples.

For the stable baselines example, the script runs for a few seconds, and then I get an exception that ultimately leads to traci.exceptions.TraCIException: Vehicle 'idm_0' is not known.

For the rllib example, the script continuously runs, but periodically I get the same exception. For the rllib example, I am not using the master branch, rather the one that fixes an issue with the observation space (PR #780).

Bug Reproduce

File in which the bug occurred: flow/examples/stable_baselines/traffic_light_grid.py and flow/examples/rllib/traffic_light_grid.py

The exact command that you did run that caused the bug: python traffic_light_grid.py from either the flow/examples/stable_baselines or flow/examples/rllib folder.

Output Log

Full error output log:

/home/alexandra/Documents/Code/Flow/flow/flow/utils/flow_warnings.py:26: PendingDeprecationWarning: The attribute minGap in SumoCarFollowingParams is deprecated, use min_gap instead.
  PendingDeprecationWarning
Warning: Vehicle 'idm_10' performs emergency braking with decel=-9.00 wished=7.50 severity=1.00, time=16.00.
Warning: Vehicle 'idm_8' performs emergency braking with decel=-9.00 wished=7.50 severity=1.00, time=18.00.
Warning: Vehicle 'idm_8' performs emergency braking with decel=-9.00 wished=7.50 severity=1.00, time=53.00.
Error: Answered with error to command 0xc4: Invalid departLane definition for vehicle 'idm_0';
 must be one of ("random", "free", "allowed", "best", "first", or an int>=0)
Error: Answered with error to command 0xc4: Vehicle 'idm_0' is not known
Traceback (most recent call last):
  File "/home/alexandra/Documents/Code/Flow/flow/flow/envs/base.py", line 486, in reset
    speed=speed)
  File "/home/alexandra/Documents/Code/Flow/flow/flow/core/kernel/vehicle/traci.py", line 1040, in add
    departSpeed=str(speed))
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/traci/_vehicle.py", line 1427, in add
    self._connection._sendExact()
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/traci/connection.py", line 106, in _sendExact
    raise TraCIException(err, prefix[1], _RESULTS[prefix[2]])
traci.exceptions.TraCIException: Invalid departLane definition for vehicle 'idm_0';
 must be one of ("random", "free", "allowed", "best", "first", or an int>=0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "examples/stable_baselines/traffic_light_grid.py", line 274, in <module>
    model = run_model(args.num_cpus, args.rollout_size, args.num_steps, args.use_inflows)
  File "examples/stable_baselines/traffic_light_grid.py", line 262, in run_model
    model.learn(total_timesteps=num_steps)
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/stable_baselines/ppo2/ppo2.py", line 334, in learn
    obs, returns, masks, actions, values, neglogpacs, states, ep_infos, true_reward = runner.run()
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/stable_baselines/ppo2/ppo2.py", line 475, in run
    self.obs[:], rewards, self.dones, infos = self.env.step(clipped_actions)
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/stable_baselines/common/vec_env/base_vec_env.py", line 134, in step
    return self.step_wait()
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/stable_baselines/common/vec_env/dummy_vec_env.py", line 44, in step_wait
    obs = self.envs[env_idx].reset()
  File "/home/alexandra/Documents/Code/Flow/flow/flow/envs/base.py", line 492, in reset
    self.k.kernel_api.vehicle.remove(veh_id)  # FIXME: hack
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/traci/_vehicle.py", line 1435, in remove
    tc.CMD_SET_VEHICLE_VARIABLE, tc.REMOVE, vehID, reason)
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/traci/connection.py", line 143, in _sendByteCmd
    self._sendExact()
  File "/home/alexandra/miniconda3/envs/flow/lib/python3.6/site-packages/traci/connection.py", line 106, in _sendExact
    raise TraCIException(err, prefix[1], _RESULTS[prefix[2]])
traci.exceptions.TraCIException: Vehicle 'idm_0' is not known

Versions

kjang96 commented 4 years ago

@acjoseph The traceback indicates that departLane isn't set correctly. This is set in the experiment script (i.e. flow/examples/stable_baselines/traffic_light_grid.py and flow/examples/rllib/traffic_light_grid.py). What are you setting your departLane to? As the traceback indicates, it should be one of ("random", "free", "allowed", "best", "first", or an int>=0)

eugenevinitsky commented 4 years ago

@kjang96 this seems to be a bigger issue; her code runs on Ubuntu 16.04 but not 18.04

eugenevinitsky commented 4 years ago

@kevin-thankyou-lin this person is having the same issue as you

kevin-thankyou-lin commented 4 years ago

Oh awesome, I’ll see if I can figure it out.

On Thu, 28 Nov 2019 at 10:53, Eugene Vinitsky notifications@github.com wrote:

@kevin-thankyou-lin https://github.com/kevin-thankyou-lin this person is having the same issue as you

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/flow-project/flow/issues/787?email_source=notifications&email_token=AH6MY6P2QLK2WVBGQNFZGXLQWAHRLA5CNFSM4JQXWCAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFNJCDY#issuecomment-559583503, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH6MY6MU7BHABRX6HZM2CMDQWAHRLANCNFSM4JQXWCAA .

acjoseph commented 4 years ago

I seem to have narrowed down the issue somewhat. If I set restart_instance = True within SumoParams, then I do not have this bug. When restart_instance = False or is not declared, I have the issue. This is still with Ubuntu 18.04.

eugenevinitsky commented 4 years ago

@kev-chien does this occur for you as well?

kev-chien commented 4 years ago

@eugenevinitsky Did you mean to reference @kevin-thankyou-lin ? If you were asking me, I don’t use Ubuntu, so I can't currently reproduce the issue, sorry!

eugenevinitsky commented 4 years ago

Yes, my bad! @kevin-thankyou-lin

eugenevinitsky commented 4 years ago

Too many kevins.

kevin-thankyou-lin commented 4 years ago

Sorry for the delay. Yes, this occurs for me as well. Nice find!

dimikout3 commented 3 years ago

Hello, did anyone solve this issue ?