search

LucasAlegre / sumo-rl

Reinforcement Learning environments for Traffic Signal Control with SUMO. Compatible with Gymnasium, PettingZoo, and popular RL libraries.
https://lucasalegre.github.io/sumo-rl
MIT License
697 stars 191 forks source link

Error while running 4x4 grid experiement #4

Closed Exorcismus closed 5 years ago

Exorcismus commented 5 years ago

Am trying to run the 4x4 grid experiment but it throws the below log


python3 experiments/a3c_4x4grid.py
2019-03-29 03:35:14,273 INFO node.py:423 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-03-29_03-35-14_19093/logs.
2019-03-29 03:35:14,379 INFO services.py:363 -- Waiting for redis server at 127.0.0.1:12017 to respond...
2019-03-29 03:35:14,505 INFO services.py:363 -- Waiting for redis server at 127.0.0.1:38753 to respond...
2019-03-29 03:35:14,510 INFO services.py:760 -- Starting Redis shard with 2.07 GB max memory.
2019-03-29 03:35:14,543 INFO services.py:1384 -- Starting the Plasma object store with 3.1 GB memory using /dev/shm.
 Retrying in 1 seconds
2019-03-29 03:35:15,693UINFO policy_evaluator.py:278 -- Creating policy evaluation worker 0 on CPU (please ignore any CUDA init errors)
2019-03-29 03:35:15.694604: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-03-29 03:35:15.708626: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2208000000 Hz
2019-03-29 03:35:15.708881: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1c98b90 executing computations on platform Host. Devices:
2019-03-29 03:35:15.708961: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/ray/rllib/models/action_dist.py:114: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.random.categorical instead.
WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
 Retrying in 1 seconds
2019-03-29 03:35:16,281 WARNING worker.py:330 -- WARNING: Falling back to serializing objects of type <class 'numpy.dtype'> by using pickle. This may be inefficient.
2019-03-29 03:35:16,282 WARNING worker.py:330 -- WARNING: Falling back to serializing objects of type <class 'mtrand.RandomState'> by using pickle. This may be inefficient.
Step #116.00 (1ms ~= 1000.00*RT, ~296000.00UPS, TraCException in thread Thread-2:
Traceback (most recent call last):
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 47, in get
    self.feed_dict, os.environ.get("TF_TIMELINE_DIR"))
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 85, in run_timeline
    fetches = sess.run(ops, feed_dict=feed_dict)
  File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (16, 11) for Tensor '0/Placeholder:0', which has shape '(?, 6)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 148, in run
    raise e
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 145, in run
    self._run()
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 167, in _run
    item = next(rollout_provider)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 316, in _env_runner
    active_episodes)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 515, in _do_policy_eval
    eval_results[k] = builder.get(v)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 50, in get
    self.fetches, self.feed_dict))
ValueError: Error fetching: [<tf.Tensor '0/Squeeze:0' shape=(?,) dtype=int64>, {'action_prob': <tf.Tensor '0/Exp_1:0' shape=(?,) dtype=float32>, 'vf_preds': <tf.Tensor '0/Reshape:0' shape=(?,) dtype=float32>}], feed_dict={<tf.Tensor '0/Placeholder:0' shape=(?, 6) dtype=float32>: [array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.9510391 , 0.9510391 , 0.95927297, 0.95927297, 0.9510391 ,
       0.9510391 ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
       0.        ]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
       0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
       0.        ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
       0.        ]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
       0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
       0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])], <tf.Tensor '0/action:0' shape=(?,) dtype=int64>: [array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)], <tf.Tensor '0/prev_reward:0' shape=(?,) dtype=float32>: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], <tf.Tensor '0/PlaceholderWithDefault:0' shape=() dtype=bool>: False}

(pid=19115)  Retrying in 1 seconds
(pid=19117)  Retrying in 1 seconds
(pid=19115) 2019-03-29 03:35:19,414     INFO policy_evaluator.py:278 -- Creating policy evaluation worker 2 on CPU (please ignore any CUDA init errors)
(pid=19115) 2019-03-29 03:35:19.416187: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
(pid=19115) 2019-03-29 03:35:19.421268: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2208000000 Hz
(pid=19115) 2019-03-29 03:35:19.421394: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x1aa5d30 executing computations on platform Host. Devices:
(pid=19115) 2019-03-29 03:35:19.421407: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
(pid=19115) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 0 ACT 0 BUF 0)           
(pid=19117) 2019-03-29 03:35:19,410     INFO policy_evaluator.py:278 -- Creating policy evaluation worker 1 on CPU (please ignore any CUDA init errors)
(pid=19117) 2019-03-29 03:35:19.413480: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
(pid=19117) 2019-03-29 03:35:19.418717: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2208000000 Hz
(pid=19117) 2019-03-29 03:35:19.418926: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x2712d30 executing computations on platform Host. Devices:
(pid=19117) 2019-03-29 03:35:19.418944: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
(pid=19117) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
(pid=19117) Instructions for updating:
(pid=19117) Colocations handled automatically by placer.
(pid=19117) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 2ms, vehicles TOT 0 ACT 0 BUF 0)           
(pid=19115) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
(pid=19115) Instructions for updating:
(pid=19115) Colocations handled automatically by placer.
(pid=19115) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/ray/rllib/models/action_dist.py:114: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
(pid=19115) Instructions for updating:
(pid=19115) Use tf.random.categorical instead.
(pid=19117) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/ray/rllib/models/action_dist.py:114: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
(pid=19117) Instructions for updating:
(pid=19117) Use tf.random.categorical instead.
(pid=19117) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
(pid=19117) Instructions for updating:
(pid=19117) Use tf.cast instead.
(pid=19115) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
(pid=19115) Instructions for updating:
(pid=19115) Use tf.cast instead.
(pid=19115) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
(pid=19115) Instructions for updating:
(pid=19115) Deprecated in favor of operator or tf.math.divide.
(pid=19117) WARNING:tensorflow:From /home/fady/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
(pid=19117) Instructions for updating:
(pid=19117) Deprecated in favor of operator or tf.math.divide.
(pid=19115)  Retrying in 1 seconds
(pid=19117)  Retrying in 1 seconds
(pid=19115) Step #0.00 (1ms ~= 1000.00*RT, ~8000.00UPS, TraCI: 11ms, vehicles TOT 8 ACT 8 
(pid=19115) Step #1.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 8 ACT 8 BUF 0)           
(pid=19115) Step #2.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 8 ACT 8 BUF 0)           
(pid=19115) Step #3.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 16 ACT 16 BUF 0)         
(pid=19115) Step #4.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 16 ACT 16 BUF 0)         
(pid=19115) Step #5.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 16 ACT 16 BUF 0)         
(pid=19115) Step #6.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 24 ACT 24 BUF 0)         
(pid=19115) Step #7.00 (1ms ~= 1000.00*RT, ~24000.00UPS, TraCI: 0ms, vehicles TOT 24 ACT 2
(pid=19115) Step #8.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 24 ACT 24 BUF 0)         
(pid=19115) Step #9.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 32 ACT 32 BUF 0)         
(pid=19115) Step #10.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 32 ACT 32 BUF 0)        
(pid=19115) Step #11.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 32 ACT 32 BUF 0)        
(pid=19115) Step #12.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 40 ACT 40 BUF 0)        
(pid=19115) Step #13.00 (1ms ~= 1000.00*RT, ~40000.00UPS, TraCI: 0ms, vehicles TOT 40 ACT 
(pid=19115) Step #14.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 40 ACT 40 BUF 0)        
(pid=19115) Step #15.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 48 ACT 48 BUF 0)        
(pid=19115) Step #16.00 (1ms ~= 1000.00*RT, ~48000.00UPS, TraCI: 0ms, vehicles TOT 48 ACT 
(pid=19115) Step #17.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 48 ACT 48 BUF 0)        
(pid=19115) Step #18.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 56 ACT 56 BUF 0)        
(pid=19115) Step #19.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 56 ACT 56 BUF 0)        
(pid=19115) Step #20.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 56 ACT 56 BUF 0)        
(pid=19115) Step #21.00 (1ms ~= 1000.00*RT, ~64000.00UPS, TraCI: 0ms, vehicles TOT 64 ACT 
(pid=19115) Step #22.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 64 ACT 64 BUF 0)        
(pid=19115) Step #23.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 64 ACT 64 BUF 0)        
(pid=19115) Step #24.00 (1ms ~= 1000.00*RT, ~72000.00UPS, TraCI: 0ms, vehicles TOT 72 ACT 
(pid=19115) Step #25.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 72 ACT 72 BUF 0)        
(pid=19115) Step #26.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 72 ACT 72 BUF 0)        
(pid=19115) Step #27.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 80 ACT 80 BUF 0)        
(pid=19115) Step #28.00 (1ms ~= 1000.00*RT, ~80000.00UPS, TraCI: 0ms, vehicles TOT 80 ACT 
(pid=19115) Step #29.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 80 ACT 80 BUF 0)        
(pid=19115) Step #30.00 (1ms ~= 1000.00*RT, ~88000.00UPS, TraCI: 0ms, vehicles TOT 88 ACT 
(pid=19115) Step #31.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 88 ACT 88 BUF 0)        
(pid=19115) Step #32.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 88 ACT 88 BUF 0)        
(pid=19115) Step #33.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 96 ACT 96 BUF 0)        
(pid=19115) Step #34.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 96 ACT 96 BUF 0)        
(pid=19115) Step #35.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 96 ACT 96 BUF 0)        
(pid=19115) Step #36.00 (1ms ~= 1000.00*RT, ~104000.00UPS, TraCI: 1ms, vehicles TOT 104 AC
(pid=19115) Step #37.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 104 ACT 104 BUF 0)      
(pid=19115) Step #38.00 (1ms ~= 1000.00*RT, ~104000.00UPS, TraCI: 0ms, vehicles TOT 104 AC
(pid=19115) Step #39.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 112 ACT 112 BUF 0)      
(pid=19115) Step #40.00 (1ms ~= 1000.00*RT, ~112000.00UPS, TraCI: 0ms, vehicles TOT 112 AC
(pid=19115) Step #41.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 112 ACT 112 BUF 0)      
(pid=19115) Step #42.00 (4ms ~= 250.00*RT, ~30000.00UPS, TraCI: 0ms, vehicles TOT 120 ACT 
(pid=19115) Step #43.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 120 ACT 120 BUF 0)      
(pid=19115) Step #44.00 (1ms ~= 1000.00*RT, ~120000.00UPS, TraCI: 0ms, vehicles TOT 120 AC
(pid=19115) Step #45.00 (1ms ~= 1000.00*RT, ~128000.00UPS, TraCI: 0ms, vehicles TOT 128 AC
(pid=19115) Step #46.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 128 ACT 128 BUF 0)      
(pid=19115) Step #47.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 128 ACT 128 BUF 0)      
(pid=19115) Step #48.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 136 ACT 136 BUF 0)      
(pid=19115) Step #49.00 (1ms ~= 1000.00*RT, ~136000.00UPS, TraCI: 0ms, vehicles TOT 136 AC
(pid=19115) Step #50.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 136 ACT 136 BUF 0)      
(pid=19115) Step #51.00 (1ms ~= 1000.00*RT, ~144000.00UPS, TraCI: 0ms, vehicles
(pid=19117) Step #0.00 (1ms ~= 1000.00*RT, ~8000.00UPS, TraCI: 5ms, vehicles TOT 8 ACT 8 B
(pid=19117) Step #1.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 8 ACT 8 BUF 0)           
(pid=19117) Step #2.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 8 ACT 8 BUF 0)           
(pid=19117) Step #3.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 16 ACT 16 BUF 0)         
(pid=19117) Step #4.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 16 ACT 16 BUF 0)         
(pid=19117) Step #5.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 16 ACT 16 BUF 0)         
(pid=19117) Step #6.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 24 ACT 24 BUF 0)         
(pid=19117) Step #7.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 24 ACT 24 BUF 0)         
(pid=19117) Step #8.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 24 ACT 24 BUF 0)         
(pid=19117) Step #9.00 (1ms ~= 1000.00*RT, ~32000.00UPS, TraCI: 1ms, vehicles TOT 32 ACT 3
(pid=19117) Step #10.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 32 ACT 32 BUF 0)        
(pid=19117) Step #11.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 32 ACT 32 BUF 0)        
(pid=19117) Step #12.00 (2ms ~= 500.00*RT, ~20000.00UPS, TraCI: 0ms, vehicles TOT 40 ACT 4
(pid=19117) Step #13.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 40 ACT 40 BUF 0)        
(pid=19117) Step #14.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 40 ACT 40 BUF 0)        
(pid=19117) Step #15.00 (2ms ~= 500.00*RT, ~24000.00UPS, TraCI: 0ms, vehicles TOT 48 ACT 4
(pid=19117) Step #16.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 48 ACT 48 BUF 0)        
(pid=19117) Step #17.00 (1ms ~= 1000.00*RT, ~48000.00UPS, TraCI: 0ms, vehicles TOT 48 ACT 
(pid=19117) Step #18.00 (2ms ~= 500.00*RT, ~28000.00UPS, TraCI: 0ms, vehicles TOT 56 ACT 5
(pid=19117) Step #19.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 56 ACT 56 BUF 0)        
(pid=19117) Step #20.00 (1ms ~= 1000.00*RT, ~56000.00UPS, TraCI: 0ms, vehicles TOT 56 ACT 
(pid=19117) Step #21.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 64 ACT 64 BUF 0)        
(pid=19117) Step #22.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 64 ACT 64 BUF 0)        
(pid=19117) Step #23.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 64 ACT 64 BUF 0)        
(pid=19117) Step #24.00 (1ms ~= 1000.00*RT, ~72000.00UPS, TraCI: 0ms, vehicles TOT 72 ACT 
(pid=19117) Step #25.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 72 ACT 72 BUF 0)        
(pid=19117) Step #26.00 (1ms ~= 1000.00*RT, ~72000.00UPS, TraCI: 0ms, vehicles TOT 72 ACT 
(pid=19117) Step #27.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 80 ACT 80 BUF 0)        
(pid=19117) Step #28.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 80 ACT 80 BUF 0)        
(pid=19117) Step #29.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 80 ACT 80 BUF 0)        
(pid=19117) Step #30.00 (1ms ~= 1000.00*RT, ~88000.00UPS, TraCI: 0ms, vehicles TOT 88 ACT 
(pid=19117) Step #31.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 88 ACT 88 BUF 0)        
(pid=19117) Step #32.00 (1ms ~= 1000.00*RT, ~88000.00UPS, TraCI: 0ms, vehicles TOT 88 ACT 
(pid=19117) Step #33.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 96 ACT 96 BUF 0)        
(pid=19117) Step #34.00 (1ms ~= 1000.00*RT, ~96000.00UPS, TraCI: 0ms, vehicles TOT 96 ACT 
(pid=19117) Step #35.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 96 ACT 96 BUF 0)        
(pid=19117) Step #36.00 (1ms ~= 1000.00*RT, ~104000.00UPS, TraCI: 0ms, vehicles TOT 104 AC
(pid=19117) Step #37.00 (1ms ~= 1000.00*RT, ~104000.00UPS, TraCI: 0ms, vehicles TOT 104 AC
(pid=19117) Step #38.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 104 ACT 104 BUF 0)      
(pid=19117) Step #39.00 (1ms ~= 1000.00*RT, ~112000.00UPS, TraCI: 0ms, vehicles TOT 112 AC
(pid=19117) Step #40.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 112 ACT 112 BUF 0)      
(pid=19117) Step #41.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 112 ACT 112 BUF 0)      
(pid=19117) Step #42.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 120 ACT 120 BUF 0)      
(pid=19117) Step #43.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 120 ACT 120 BUF 0)      
(pid=19117) Step #44.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 120 ACT 120 BUF 0)      
(pid=19117) Step #45.00 (1ms ~= 1000.00*RT, ~128000.00UPS, TraCI: 0ms, vehicles TOT 128 AC
(pid=19117) Step #46.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 128 ACT 128 BUF 0)      
(pid=19117) Step #47.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 128 ACT 128 BUF 0)      
(pid=19117) Step #48.00 (1ms ~= 1000.00*RT, ~136000.00UPS, TraCI: 0ms, vehicles TOT 136 AC
(pid=19117) Step #49.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 136 ACT 136 BUF 0)      
(pid=19117) Step #50.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 136 ACT 136 BUF 0)      
(pid=19117) Step #51.00 (1ms ~= 1000.00*RT, ~144000.00UPS, TraCI: 0ms, vehicles
(pid=19115)  TOT 144 AC
(pid=19115) Step #52.00 (1ms ~= 1000.00*RT, ~144000.00UPS, TraCI: 0ms, vehicles TOT 144 AC
(pid=19115) Step #53.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 144 ACT 144 BUF 0)      
(pid=19115) Step #54.00 (1ms ~= 1000.00*RT, ~152000.00UPS, TraCI: 0ms, vehicles TOT 152 AC
(pid=19115) Step #55.00 (1ms ~= 1000.00*RT, ~152000.00UPS, TraCI: 0ms, vehicles TOT 152 AC
(pid=19115) Step #56.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 152 ACT 152 BUF 0)      
(pid=19115) Step #57.00 (1ms ~= 1000.00*RT, ~160000.00UPS, TraCI: 0ms, vehicles TOT 160 AC
(pid=19115) Step #58.00 (1ms ~= 1000.00*RT, ~160000.00UPS, TraCI: 0ms, vehicles TOT 160 AC
(pid=19115) Step #59.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 160 ACT 160 BUF 0)      
(pid=19115) Step #60.00 (1ms ~= 1000.00*RT, ~168000.00UPS, TraCI: 0ms, vehicles TOT 168 AC
(pid=19115) Step #61.00 (1ms ~= 1000.00*RT, ~168000.00UPS, TraCI: 0ms, vehicles TOT 168 AC
(pid=19115) Step #62.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 168 ACT 168 BUF 0)      
(pid=19115) Step #63.00 (1ms ~= 1000.00*RT, ~176000.00UPS, TraCI: 0ms, vehicles TOT 176 AC
(pid=19115) Step #64.00 (1ms ~= 1000.00*RT, ~176000.00UPS, TraCI: 0ms, vehicles TOT 176 AC
(pid=19115) Step #65.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 176 ACT 176 BUF 0)      
(pid=19115) Step #66.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 184 ACT 184 BUF 0)      
(pid=19115) Step #67.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 184 ACT 184 BUF 0)      
(pid=19115) Step #68.00 (1ms ~= 1000.00*RT, ~184000.00UPS, TraCI: 0ms, vehicles TOT 184 AC
(pid=19115) Step #69.00 (1ms ~= 1000.00*RT, ~192000.00UPS, TraCI: 0ms, vehicles TOT 192 AC
(pid=19115) Step #70.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 192 ACT 192 BUF 0)      
(pid=19115) Step #71.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 192 ACT 192 BUF 0)      
(pid=19115) Step #72.00 (1ms ~= 1000.00*RT, ~200000.00UPS, TraCI: 0ms, vehicles TOT 200 AC
(pid=19115) Step #73.00 (1ms ~= 1000.00*RT, ~200000.00UPS, TraCI: 0ms, vehicles TOT 200 AC
(pid=19115) Step #74.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 200 ACT 200 BUF 0)      
(pid=19115) Step #75.00 (1ms ~= 1000.00*RT, ~208000.00UPS, TraCI: 0ms, vehicles TOT 208 AC
(pid=19115) Step #76.00 (1ms ~= 1000.00*RT, ~208000.00UPS, TraCI: 0ms, vehicles TOT 208 AC
(pid=19115) Step #77.00 (1ms ~= 1000.00*RT, ~208000.00UPS, TraCI: 0ms, vehicles TOT 208 AC
(pid=19115) Step #78.00 (1ms ~= 1000.00*RT, ~216000.00UPS, TraCI: 0ms, vehicles TOT 216 AC
(pid=19115) Step #79.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 216 ACT 216 BUF 0)      
(pid=19115) Step #80.00 (1ms ~= 1000.00*RT, ~216000.00UPS, TraCI: 0ms, vehicles TOT 216 AC
(pid=19115) Step #81.00 (1ms ~= 1000.00*RT, ~224000.00UPS, TraCI: 0ms, vehicles TOT 224 AC
(pid=19115) Step #82.00 (1ms ~= 1000.00*RT, ~224000.00UPS, TraCI: 0ms, vehicles TOT 224 AC
(pid=19115) Step #83.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 224 ACT 224 BUF 0)      
(pid=19115) Step #84.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 232 ACT 232 BUF 0)      
(pid=19115) Step #85.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 232 ACT 232 BUF 0)      
(pid=19115) Step #86.00 (1ms ~= 1000.00*RT, ~232000.00UPS, TraCI: 0ms, vehicles TOT 232 AC
(pid=19115) Step #87.00 (1ms ~= 1000.00*RT, ~240000.00UPS, TraCI: 0ms, vehicles TOT 240 AC
(pid=19115) Step #88.00 (1ms ~= 1000.00*RT, ~240000.00UPS, TraCI: 0ms, vehicles TOT 240 AC
(pid=19115) Step #89.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 240 ACT 240 BUF 0)      
(pid=19115) Step #90.00 (1ms ~= 1000.00*RT, ~248000.00UPS, TraCI: 1ms, vehicles TOT 248 AC
(pid=19115) Step #91.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 248 ACT 248 BUF 0)      
(pid=19115) Step #92.00 (1ms ~= 1000.00*RT, ~248000.00UPS, TraCI: 0ms, vehicles TOT 248 AC
(pid=19115) Step #93.00 (1ms ~= 1000.00*RT, ~256000.00UPS, TraCI: 0ms, vehicles TOT 256 AC
(pid=19115) Step #94.00 (1ms ~= 1000.00*RT, ~256000.00UPS, TraCI: 0ms, vehicles TOT 256 AC
(pid=19115) Step #95.00 (1ms ~= 1000.00*RT, ~256000.00UPS, TraCI: 0ms, vehicles TOT 256 AC
(pid=19115) Step #96.00 (1ms ~= 1000.00*RT, ~264000.00UPS, TraCI: 0ms, vehicles TOT 264 AC
(pid=19115) Step #97.00 (1ms ~= 1000.00*RT, ~264000.00UPS, TraCI: 0ms, vehicles TOT 264 AC
(pid=19115) Step #98.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 264 ACT 264 BUF 0)      
(pid=19115) Step #99.00 (1ms ~= 1000.00*RT, ~272000.00UPS, TraCI: 0ms, vehicles TOT 272 AC
(pid=19115) Step #100.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 272 ACT 272 BUF 0)     
(pid=19115) Step #101.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 272 ACT 272 BUF 0)     
(pid=19115) Step #102.00 (1ms ~= 1000.00*RT, ~280000.00UPS, TraCI: 0ms, vehicles TOT 280 A
(pid=19115) Step #103.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 
(pid=19117) Exception in thread Thread-1:
(pid=19117) Traceback (most recent call last):
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 47, in get
(pid=19117)     self.feed_dict, os.environ.get("TF_TIMELINE_DIR"))
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 85, in run_timeline
(pid=19117)     fetches = sess.run(ops, feed_dict=feed_dict)
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
(pid=19117)     run_metadata_ptr)
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
(pid=19117)     str(subfeed_t.get_shape())))
(pid=19117) ValueError: Cannot feed value of shape (16, 11) for Tensor '0/Placeholder:0', which has shape '(?, 6)'
(pid=19117) 
(pid=19117) During handling of the above exception, another exception occurred:
(pid=19117) 
(pid=19117) Traceback (most recent call last):
(pid=19117)   File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
(pid=19117)     self.run()
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 148, in run
(pid=19117)     raise e
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 145, in run
(pid=19117)     self._run()
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 167, in _run
(pid=19117)     item = next(rollout_provider)
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 316, in _env_runner
(pid=19117)     active_episodes)
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 515, in _do_policy_eval
(pid=19117)     eval_results[k] = builder.get(v)
(pid=19117)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 50, in get
(pid=19117)     self.fetches, self.feed_dict))
(pid=19117) ValueError: Error fetching: [<tf.Tensor '0/Squeeze:0' shape=(?,) dtype=int64>, {'action_prob': <tf.Tensor '0/Exp_1:0' shape=(?,) dtype=float32>, 'vf_preds': <tf.Tensor '0/Reshape:0' shape=(?,) dtype=float32>}], feed_dict={<tf.Tensor '0/Placeholder:0' shape=(?, 6) dtype=float32>: [array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19117)        0.9510391 , 0.9510391 , 0.95927297, 0.95927297, 0.9510391 ,
(pid=19117)        0.9510391 ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19117)        0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
(pid=19117)        0.        ]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
(pid=19117)        0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19117)        0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
(pid=19117)        0.        ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19117)        0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
(pid=19117)        0.        ]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
(pid=19117)        0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
(pid=19117)        0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])], <tf.Tensor '0/action:0' shape=(?,) dtype=int64>: [array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)], <tf.Tensor '0/prev_reward:0' shape=(?,) dtype=float32>: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], <tf.Tensor '0/PlaceholderWithDefault:0' shape=() dtype=bool>: False}
(pid=19117) 
(pid=19117)  TOT 144 AC
(pid=19117) Step #52.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 144 ACT 144 BUF 0)      
(pid=19117) Step #53.00 (1ms ~= 1000.00*RT, ~144000.00UPS, TraCI: 0ms, vehicles TOT 144 AC
(pid=19117) Step #54.00 (3ms ~= 333.33*RT, ~50666.67UPS, TraCI: 0ms, vehicles TOT 152 ACT 
(pid=19117) Step #55.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 152 ACT 152 BUF 0)      
(pid=19117) Step #56.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 152 ACT 152 BUF 0)      
(pid=19117) Step #57.00 (1ms ~= 1000.00*RT, ~160000.00UPS, TraCI: 0ms, vehicles TOT 160 AC
(pid=19117) Step #58.00 (1ms ~= 1000.00*RT, ~160000.00UPS, TraCI: 0ms, vehicles TOT 160 AC
(pid=19117) Step #59.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 160 ACT 160 BUF 0)      
(pid=19117) Step #60.00 (1ms ~= 1000.00*RT, ~168000.00UPS, TraCI: 0ms, vehicles TOT 168 AC
(pid=19117) Step #61.00 (1ms ~= 1000.00*RT, ~168000.00UPS, TraCI: 0ms, vehicles TOT 168 AC
(pid=19117) Step #62.00 (1ms ~= 1000.00*RT, ~168000.00UPS, TraCI: 0ms, vehicles TOT 168 AC
(pid=19117) Step #63.00 (1ms ~= 1000.00*RT, ~176000.00UPS, TraCI: 0ms, vehicles TOT 176 AC
(pid=19117) Step #64.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 176 ACT 176 BUF 0)      
(pid=19117) Step #65.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 176 ACT 176 BUF 0)      
(pid=19117) Step #66.00 (1ms ~= 1000.00*RT, ~184000.00UPS, TraCI: 0ms, vehicles TOT 184 AC
(pid=19117) Step #67.00 (1ms ~= 1000.00*RT, ~184000.00UPS, TraCI: 0ms, vehicles TOT 184 AC
(pid=19117) Step #68.00 (1ms ~= 1000.00*RT, ~184000.00UPS, TraCI: 0ms, vehicles TOT 184 AC
(pid=19117) Step #69.00 (1ms ~= 1000.00*RT, ~192000.00UPS, TraCI: 0ms, vehicles TOT 192 AC
(pid=19117) Step #70.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 192 ACT 192 BUF 0)      
(pid=19117) Step #71.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 192 ACT 192 BUF 0)      
(pid=19117) Step #72.00 (1ms ~= 1000.00*RT, ~200000.00UPS, TraCI: 0ms, vehicles TOT 200 AC
(pid=19117) Step #73.00 (1ms ~= 1000.00*RT, ~200000.00UPS, TraCI: 0ms, vehicles TOT 200 AC
(pid=19117) Step #74.00 (1ms ~= 1000.00*RT, ~200000.00UPS, TraCI: 0ms, vehicles TOT 200 AC
(pid=19117) Step #75.00 (1ms ~= 1000.00*RT, ~208000.00UPS, TraCI: 0ms, vehicles TOT 208 AC
(pid=19117) Step #76.00 (1ms ~= 1000.00*RT, ~208000.00UPS, TraCI: 0ms, vehicles TOT 208 AC
(pid=19117) Step #77.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 208 ACT 208 BUF 0)      
(pid=19117) Step #78.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 216 ACT 216 BUF 0)      
(pid=19117) Step #79.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 216 ACT 216 BUF 0)      
(pid=19117) Step #80.00 (1ms ~= 1000.00*RT, ~216000.00UPS, TraCI: 0ms, vehicles TOT 216 AC
(pid=19117) Step #81.00 (1ms ~= 1000.00*RT, ~224000.00UPS, TraCI: 0ms, vehicles TOT 224 AC
(pid=19117) Step #82.00 (1ms ~= 1000.00*RT, ~224000.00UPS, TraCI: 0ms, vehicles TOT 224 AC
(pid=19117) Step #83.00 (1ms ~= 1000.00*RT, ~224000.00UPS, TraCI: 0ms, vehicles TOT 224 AC
(pid=19117) Step #84.00 (1ms ~= 1000.00*RT, ~232000.00UPS, TraCI: 0ms, vehicles TOT 232 AC
(pid=19117) Step #85.00 (0ms ?*RT. ?UPS, TraCI: 0ms, vehicles TOT 232 ACT 232 BUF 0)      
(pid=19117) Step #86.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 232 ACT 232 BUF 0)      
(pid=19117) Step #87.00 (1ms ~= 1000.00*RT, ~240000.00UPS, TraCI: 0ms, vehicles TOT 240 AC
(pid=19117) Step #88.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 240 ACT 240 BUF 0)      
(pid=19117) Step #89.00 (1ms ~= 1000.00*RT, ~240000.00UPS, TraCI: 0ms, vehicles TOT 240 AC
(pid=19117) Step #90.00 (1ms ~= 1000.00*RT, ~248000.00UPS, TraCI: 0ms, vehicles TOT 248 AC
(pid=19117) Step #91.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 248 ACT 248 BUF 0)      
(pid=19117) Step #92.00 (1ms ~= 1000.00*RT, ~248000.00UPS, TraCI: 0ms, vehicles TOT 248 AC
(pid=19117) Step #93.00 (1ms ~= 1000.00*RT, ~256000.00UPS, TraCI: 0ms, vehicles TOT 256 AC
(pid=19117) Step #94.00 (1ms ~= 1000.00*RT, ~256000.00UPS, TraCI: 0ms, vehicles TOT 256 AC
(pid=19117) Step #95.00 (1ms ~= 1000.00*RT, ~256000.00UPS, TraCI: 0ms, vehicles TOT 256 AC
(pid=19117) Step #96.00 (1ms ~= 1000.00*RT, ~264000.00UPS, TraCI: 0ms, vehicles TOT 264 AC
(pid=19117) Step #97.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 264 ACT 264 BUF 0)      
(pid=19117) Step #98.00 (1ms ~= 1000.00*RT, ~264000.00UPS, TraCI: 0ms, vehicles TOT 264 AC
(pid=19117) Step #99.00 (1ms ~= 1000.00*RT, ~272000.00UPS, TraCI: 0ms, vehicles TOT 272 AC
(pid=19117) Step #100.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 272 ACT 272 BUF 0)     
(pid=19117) Step #101.00 (1ms ~= 1000.00*RT, ~272000.00UPS, TraCI: 0ms, vehicles TOT 272 A
(pid=19117) Step #102.00 (1ms ~= 1000.00*RT, ~280000.00UPS, TraCI: 0ms, vehicles TOT 280 A
(pid=19117) Step #103.00 (0ms ?*RT. ?UPS, TraCI: 1ms, vehicles TOT 
2019-03-29 03:35:21,046 INFO agent.py:313 -- Worker crashed during call to train(). To attempt to continue training without the failed worker, set `'ignore_worker_failures': True`.
(pid=19115) Exception in thread Thread-1:
(pid=19115) Traceback (most recent call last):
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 47, in get
(pid=19115)     self.feed_dict, os.environ.get("TF_TIMELINE_DIR"))
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 85, in run_timeline
(pid=19115)     fetches = sess.run(ops, feed_dict=feed_dict)
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
(pid=19115)     run_metadata_ptr)
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
(pid=19115)     str(subfeed_t.get_shape())))
(pid=19115) ValueError: Cannot feed value of shape (16, 11) for Tensor '0/Placeholder:0', which has shape '(?, 6)'
(pid=19115) 
(pid=19115) During handling of the above exception, another exception occurred:
(pid=19115) 
(pid=19115) Traceback (most recent call last):
(pid=19115)   File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
(pid=19115)     self.run()
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 148, in run
(pid=19115)     raise e
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 145, in run
(pid=19115)     self._run()
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 167, in _run
(pid=19115)     item = next(rollout_provider)
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 316, in _env_runner
(pid=19115)     active_episodes)
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 515, in _do_policy_eval
(pid=19115)     eval_results[k] = builder.get(v)
(pid=19115)   File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 50, in get
(pid=19115)     self.fetches, self.feed_dict))
(pid=19115) ValueError: Error fetching: [<tf.Tensor '0/Squeeze:0' shape=(?,) dtype=int64>, {'action_prob': <tf.Tensor '0/Exp_1:0' shape=(?,) dtype=float32>, 'vf_preds': <tf.Tensor '0/Reshape:0' shape=(?,) dtype=float32>}], feed_dict={<tf.Tensor '0/Placeholder:0' shape=(?, 6) dtype=float32>: [array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19115)        0.9510391 , 0.9510391 , 0.95927297, 0.95927297, 0.9510391 ,
(pid=19115)        0.9510391 ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19115)        0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
(pid=19115)        0.        ]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
(pid=19115)        0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19115)        0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
(pid=19115)        0.        ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
(pid=19115)        0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
(pid=19115)        0.        ]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
(pid=19115)        0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
(pid=19115)        0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])], <tf.Tensor '0/action:0' shape=(?,) dtype=int64>: [array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)], <tf.Tensor '0/prev_reward:0' shape=(?,) dtype=float32>: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], <tf.Tensor '0/PlaceholderWithDefault:0' shape=() dtype=bool>: False}
(pid=19115) 
Traceback (most recent call last):
  File "experiments/a3c_4x4grid.py", line 49, in <module>
    print(trainer.train())  # distributed training step
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/agents/agent.py", line 316, in train
    raise e
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/agents/agent.py", line 305, in train
    result = Trainable.train(self)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/tune/trainable.py", line 151, in train
    result = self._train()
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/agents/a3c/a3c.py", line 71, in _train
    self.optimizer.step()
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/optimizers/async_gradients_optimizer.py", line 50, in step
    gradient, info = ray.get(future)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/worker.py", line 2316, in get
    raise value
ray.exceptions.RayTaskError: ray_PolicyEvaluator:compute_gradients() (pid=19117, host=dellg5)
ray.exceptions.RayTaskError: ray_PolicyEvaluator:sample() (pid=19117, host=dellg5)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 47, in get
    self.feed_dict, os.environ.get("TF_TIMELINE_DIR"))
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 85, in run_timeline
    fetches = sess.run(ops, feed_dict=feed_dict)
  File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/fady/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (16, 11) for Tensor '0/Placeholder:0', which has shape '(?, 6)'

During handling of the above exception, another exception occurred:

ray_PolicyEvaluator:sample() (pid=19117, host=dellg5)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/policy_evaluator.py", line 393, in sample
    batches = [self.input_reader.next()]
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 41, in next
    batches = [self.get_data()]
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 180, in get_data
    raise rollout
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 148, in run
    raise e
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 145, in run
    self._run()
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 167, in _run
    item = next(rollout_provider)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 316, in _env_runner
    active_episodes)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 515, in _do_policy_eval
    eval_results[k] = builder.get(v)
  File "/home/fady/.local/lib/python3.6/site-packages/ray/rllib/utils/tf_run_builder.py", line 50, in get
    self.fetches, self.feed_dict))
ValueError: Error fetching: [<tf.Tensor '0/Squeeze:0' shape=(?,) dtype=int64>, {'action_prob': <tf.Tensor '0/Exp_1:0' shape=(?,) dtype=float32>, 'vf_preds': <tf.Tensor '0/Reshape:0' shape=(?,) dtype=float32>}], feed_dict={<tf.Tensor '0/Placeholder:0' shape=(?, 6) dtype=float32>: [array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.9510391 , 0.9510391 , 0.95927297, 0.95927297, 0.9510391 ,
       0.9510391 ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
       0.        ]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
       0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
       0.        ]), array([1.        , 0.        , 0.        , 0.95927297, 0.95927297,
       0.        , 0.        , 0.95927297, 0.95927297, 0.        ,
       0.        ]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
       0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), array([1.       , 0.       , 0.       , 0.       , 0.       , 0.9510391,
       0.9510391, 0.       , 0.       , 0.9510391, 0.9510391]), array([1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])], <tf.Tensor '0/action:0' shape=(?,) dtype=int64>: [array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0), array(0)], <tf.Tensor '0/prev_reward:0' shape=(?,) dtype=float32>: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], <tf.Tensor '0/PlaceholderWithDefault:0' shape=() dtype=bool>: False}

Step #120.00Error: tcpip::Socket::recvAndCheck @ recv: peer shutdown TOT 296 A
Quitting (on error).
LucasAlegre commented 5 years ago

The observation space on this experiment was wrong. I fixed it in my last commit, it should work now. Thanks!