Please add more tutorials for multiagent examples

Thank you @eugenevinitsky for the reopening. I followed the instruction of @ashkan-software and ran the multiagent example, but I think the result is a little wired.

First I ran multiagent_highway example without any change but change N_CPUS = 1, for my pc only have one cpu. I think theoretically the whole training process will not end until it runs out of the set ITERATIONS. However it stopped after couple of epochs.

Then I set sim=SumoParams(render=True) trying to see what is going on on sumo gui. While every time I hit the play button it pop out a new window with a static network. 2019-12-29 11-10-46屏幕截图

After clicking four or five times of the play button, it showed the dynamic training process around time 100. 2019-12-29 11-11-53屏幕截图

Here is a part of the terminal outputs, it showed the same outputs repeatedly

2019-12-29 10:34:07,146 INFO trial_runner.py:587 -- Attempting to recover trial state from last checkpoint. 2019-12-29 10:34:07,159 WARNING signature.py:108 -- The function with_updates has a **kwargs argument, which is currently not supported. (pid=1169) 2019-12-29 10:34:07,138 INFO trainer.py:366 -- Worker crashed during call to train(). To attempt to continue training without the failed worker, set 'ignore_worker_failures': True. == Status == Using FIFO scheduling algorithm. Resources requested: 2/2 CPUs, 0/1 GPUs Memory usage on this node: 5.5/8.3 GB Result logdir: /home/elizabeth/ray_results/stabilizing_open_network_merges Number of trials: 1 ({'RUNNING': 1}) RUNNING trials:

PPO_MergePOEnv-v0_0: RUNNING, 2 failures: /home/elizabeth/ray_results/stabilizing_open_network_merges/PPO_MergePOEnv-v0_0_2019-12-29_10-33-102tsagrr5/error_2019-12-29_10-34-07.txt

(pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/compat/v2_compat.py:61: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) non-resource variables are not supported in the long term (pid=1279) 2019-12-29 10:34:10,840 WARNING ppo.py:143 -- FYI: By default, the value function will not share layers with the policy model ('vf_share_layers': False). (pid=1279) 2019-12-29 10:34:14,448 INFO rollout_worker.py:319 -- Creating policy evaluation worker 0 on CPU (please ignore any CUDA init errors) (pid=1279) 2019-12-29 10:34:14.451842: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA (pid=1279) 2019-12-29 10:34:14.482045: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz (pid=1279) 2019-12-29 10:34:14.482510: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5f2cf70 executing computations on platform Host. Devices: (pid=1279) 2019-12-29 10:34:14.482575: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/fcnet_v1.py:48: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Use keras.layers.dense instead. (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/fcnet_v1.py:48: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Use keras.layers.dense instead. (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cc88>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cc88>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cc88>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cc88>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8ccc0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cd30>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cd30>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cd30>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43eaa8cd30>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/tf_action_dist.py:138: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Use tf.cast instead. (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/tf_action_dist.py:138: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Use tf.cast instead. (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797320>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797320>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797320>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797320>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797e80>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797e80>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797e80>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c0797e80>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43c07826d8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) 2019-12-29 10:34:15.636531: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile. (pid=1279) 2019-12-29 10:34:15,660 INFO dynamic_tf_policy.py:324 -- Initializing loss function with dummy input: (pid=1279) (pid=1279) { 'action_prob': <tf.Tensor 'default_policy/action_prob:0' shape=(?,) dtype=float32>, (pid=1279) 'actions': <tf.Tensor 'default_policy/actions:0' shape=(?, 5) dtype=float32>, (pid=1279) 'advantages': <tf.Tensor 'default_policy/advantages:0' shape=(?,) dtype=float32>, (pid=1279) 'behaviour_logits': <tf.Tensor 'default_policy/behaviour_logits:0' shape=(?, 10) dtype=float32>, (pid=1279) 'dones': <tf.Tensor 'default_policy/dones:0' shape=(?,) dtype=bool>, (pid=1279) 'new_obs': <tf.Tensor 'default_policy/new_obs:0' shape=(?, 25) dtype=float32>, (pid=1279) 'obs': <tf.Tensor 'default_policy/observation:0' shape=(?, 25) dtype=float32>, (pid=1279) 'prev_actions': <tf.Tensor 'default_policy/action:0' shape=(?, 5) dtype=float32>, (pid=1279) 'prev_rewards': <tf.Tensor 'default_policy/prev_reward:0' shape=(?,) dtype=float32>, (pid=1279) 'rewards': <tf.Tensor 'default_policy/rewards:0' shape=(?,) dtype=float32>, (pid=1279) 'value_targets': <tf.Tensor 'default_policy/value_targets:0' shape=(?,) dtype=float32>, (pid=1279) 'vf_preds': <tf.Tensor 'default_policy/vf_preds:0' shape=(?,) dtype=float32>} (pid=1279) (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Use tf.where in 2.0, which has the same broadcast rule as np.where (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Use tf.where in 2.0, which has the same broadcast rule as np.where (pid=1279) 2019-12-29 10:34:16,926 INFO rollout_worker.py:742 -- Built policy map: {'default_policy': <ray.rllib.policy.tf_policy_template.PPOTFPolicy object at 0x7f43eaa8c128>} (pid=1279) 2019-12-29 10:34:16,926 INFO rollout_worker.py:743 -- Built preprocessor map: {'default_policy': <ray.rllib.models.preprocessors.NoPreprocessor object at 0x7f43eaa8c048>} (pid=1279) 2019-12-29 10:34:16,926 INFO rollout_worker.py:356 -- Built filter map: {'default_policy': <ray.rllib.utils.filter.NoFilter object at 0x7f43eaa7feb8>} (pid=1279) 2019-12-29 10:34:16,955 INFO multi_gpu_optimizer.py:93 -- LocalMultiGPUOptimizer devices ['/cpu:0'] (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721ef0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721ef0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721ef0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721ef0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721f28>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721eb8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721eb8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721eb8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4721eb8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a467dc18>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a467dc18>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a467dc18>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a467dc18>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4735320>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4735320>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4735320>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4735320>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46c4b00>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46c4b00>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46c4b00>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46c4b00>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4696128>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4696128>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4696128>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4696128>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a405a128>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a405a128>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a405a128>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a405a128>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4047f28>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4047f28>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4047f28>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4047f28>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/compat/v2_compat.py:61: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) non-resource variables are not supported in the long term (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a407c7b8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2cf8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2cf8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2cf8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2cf8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2438>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2438>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2438>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43877d2438>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46dc860>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46dc860>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46dc860>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a46dc860>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4061978>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4061978>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4061978>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f43a4061978>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1279) 2019-12-29 10:34:23,100 INFO trainable.py:105 -- _setup took 12.486 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads. (pid=1279) 2019-12-29 10:34:23,100 WARNING util.py:47 -- Install gputil for GPU system monitoring. (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/policy/tf_policy.py:570: Variable.load (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Prefer Variable.assign which has equivalent behavior in 2.X. (pid=1279) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/policy/tf_policy.py:570: Variable.load (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. (pid=1279) Instructions for updating: (pid=1279) Prefer Variable.assign which has equivalent behavior in 2.X. (pid=1325) 2019-12-29 10:34:23,642 INFO rollout_worker.py:319 -- Creating policy evaluation worker 1 on CPU (please ignore any CUDA init errors) (pid=1325) 2019-12-29 10:34:23.667857: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA (pid=1325) 2019-12-29 10:34:23.672795: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz (pid=1325) 2019-12-29 10:34:23.673068: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x70320e0 executing computations on platform Host. Devices: (pid=1325) 2019-12-29 10:34:23.673098: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/fcnet_v1.py:48: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Use keras.layers.dense instead. (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/fcnet_v1.py:48: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Use keras.layers.dense instead. (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac3c8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac3c8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac3c8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac3c8>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac470>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac470>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac470>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac470>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c402860>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c402860>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c402860>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c402860>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c3ec390>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c3ec390>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c3ec390>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c3ec390>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/tf_action_dist.py:138: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Use tf.cast instead. (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/tf/tf_action_dist.py:138: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Use tf.cast instead. (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feacc50>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feacc50>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feacc50>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feacc50>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917feac908>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c4015c0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c4015c0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c4015c0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f917c4015c0>>: AssertionError: Bad argument number for Name: 3, expecting 4 (pid=1325) 2019-12-29 10:34:24.851197: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile. (pid=1325) 2019-12-29 10:34:24,875 INFO dynamic_tf_policy.py:324 -- Initializing loss function with dummy input: (pid=1325) (pid=1325) { 'action_prob': <tf.Tensor 'default_policy/action_prob:0' shape=(?,) dtype=float32>, (pid=1325) 'actions': <tf.Tensor 'default_policy/actions:0' shape=(?, 5) dtype=float32>, (pid=1325) 'advantages': <tf.Tensor 'default_policy/advantages:0' shape=(?,) dtype=float32>, (pid=1325) 'behaviour_logits': <tf.Tensor 'default_policy/behaviour_logits:0' shape=(?, 10) dtype=float32>, (pid=1325) 'dones': <tf.Tensor 'default_policy/dones:0' shape=(?,) dtype=bool>, (pid=1325) 'new_obs': <tf.Tensor 'default_policy/new_obs:0' shape=(?, 25) dtype=float32>, (pid=1325) 'obs': <tf.Tensor 'default_policy/observation:0' shape=(?, 25) dtype=float32>, (pid=1325) 'prev_actions': <tf.Tensor 'default_policy/action:0' shape=(?, 5) dtype=float32>, (pid=1325) 'prev_rewards': <tf.Tensor 'default_policy/prev_reward:0' shape=(?,) dtype=float32>, (pid=1325) 'rewards': <tf.Tensor 'default_policy/rewards:0' shape=(?,) dtype=float32>, (pid=1325) 'value_targets': <tf.Tensor 'default_policy/value_targets:0' shape=(?,) dtype=float32>, (pid=1325) 'vf_preds': <tf.Tensor 'default_policy/vf_preds:0' shape=(?,) dtype=float32>} (pid=1325) (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Use tf.where in 2.0, which has the same broadcast rule as np.where (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Use tf.where in 2.0, which has the same broadcast rule as np.where (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/policy/tf_policy.py:570: Variable.load (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Prefer Variable.assign which has equivalent behavior in 2.X. (pid=1325) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/policy/tf_policy.py:570: Variable.load (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. (pid=1325) Instructions for updating: (pid=1325) Prefer Variable.assign which has equivalent behavior in 2.X. (pid=1325) 2019-12-29 10:34:26,177 INFO rollout_worker.py:451 -- Generating sample batch of size 200 (pid=1325) 2019-12-29 10:34:28,273 INFO sampler.py:304 -- Raw obs from env: { 0: { 'agent0': np.ndarray((25,), dtype=int64, min=0.0, max=0.0, mean=0.0)}} (pid=1325) 2019-12-29 10:34:28,274 INFO sampler.py:305 -- Info return from env: {0: {'agent0': None}} (pid=1325) 2019-12-29 10:34:28,277 INFO sampler.py:403 -- Preprocessed obs: np.ndarray((25,), dtype=int64, min=0.0, max=0.0, mean=0.0) (pid=1325) 2019-12-29 10:34:28,277 INFO sampler.py:407 -- Filtered obs: np.ndarray((25,), dtype=int64, min=0.0, max=0.0, mean=0.0) (pid=1325) 2019-12-29 10:34:28,278 INFO sampler.py:521 -- Inputs to compute_actions(): (pid=1325) (pid=1325) { 'default_policy': [ { 'data': { 'agent_id': 'agent0', (pid=1325) 'env_id': 0, (pid=1325) 'info': None, (pid=1325) 'obs': np.ndarray((25,), dtype=int64, min=0.0, max=0.0, mean=0.0), (pid=1325) 'prev_action': np.ndarray((5,), dtype=float32, min=0.0, max=0.0, mean=0.0), (pid=1325) 'prev_reward': 0.0, (pid=1325) 'rnn_state': []}, (pid=1325) 'type': 'PolicyEvalData'}]} (pid=1325) (pid=1325) 2019-12-29 10:34:28,278 INFO tf_run_builder.py:92 -- Executing TF run without tracing. To dump TF timeline traces to disk, set the TF_TIMELINE_DIR environment variable. (pid=1325) 2019-12-29 10:34:28,327 INFO sampler.py:548 -- Outputs of compute_actions(): (pid=1325) (pid=1325) { 'default_policy': ( np.ndarray((1, 5), dtype=float32, min=-2.137, max=0.31, mean=-0.816), (pid=1325) [], (pid=1325) { 'action_prob': np.ndarray((1,), dtype=float32, min=0.0, max=0.0, mean=0.0), (pid=1325) 'behaviour_logits': np.ndarray((1, 10), dtype=float32, min=0.0, max=0.0, mean=0.0), (pid=1325) 'vf_preds': np.ndarray((1,), dtype=float32, min=0.0, max=0.0, mean=0.0)})} (pid=1325) 2019-12-29 10:34:31,351 ERROR trial_runner.py:550 -- Error processing event. Traceback (most recent call last): File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 498, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 342, in fetch_result result = ray.get(trial_future[0]) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/worker.py", line 2247, in get raise value ray.exceptions.RayTaskError: ray_PPO:train() (pid=1279, host=ins7460) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 369, in train raise e File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 358, in train result = Trainable.train(self) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/tune/trainable.py", line 171, in train result = self._train() File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 126, in _train fetches = self.optimizer.step() File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/optimizers/multi_gpu_optimizer.py", line 140, in step self.num_envs_per_worker, self.train_batch_size) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/optimizers/rollout.py", line 29, in collect_samples next_sample = ray_get_and_free(fut_sample) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/utils/memory.py", line 33, in ray_get_and_free result = ray.get(object_ids) ray.exceptions.RayTaskError: ray_RolloutWorker:sample() (pid=1325, host=ins7460) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 453, in sample batches = [self.input_reader.next()] File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 56, in next batches = [self.get_data()] File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 97, in get_data item = next(self.rollout_provider) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 313, in _env_runner soft_horizon) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 401, in _process_observations policy_id).transform(raw_obs) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/preprocessors.py", line 166, in transform self.check_shape(observation) File "/home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/ray/rllib/models/preprocessors.py", line 61, in check_shape self._obs_space, observation) ValueError: ('Observation outside expected value range', Box(25,), array([ 0.13033402, 0.86966598, 1. , 0.04617912, 0.00612267, 0.16956682, -0.00946337, 0.16952428, 0.06460617, 0.00908585, 0.04033486, 0.00979989, 0.00459936, -0.03016351, 0.00443692,

, 0. , 0. , 0. , 0. ,

, 0. , 0. , 0. , 0. ]))

2019-12-29 10:34:31,356 INFO trial_runner.py:587 -- Attempting to recover trial state from last checkpoint. 2019-12-29 10:34:31,371 WARNING signature.py:108 -- The function with_updates has a **kwargs argument, which is currently not supported. (pid=1279) 2019-12-29 10:34:31,348 INFO trainer.py:366 -- Worker crashed during call to train(). To attempt to continue training without the failed worker, set 'ignore_worker_failures': True. == Status == Using FIFO scheduling algorithm. Resources requested: 2/2 CPUs, 0/1 GPUs Memory usage on this node: 5.5/8.3 GB Result logdir: /home/elizabeth/ray_results/stabilizing_open_network_merges Number of trials: 1 ({'RUNNING': 1}) RUNNING trials:

PPO_MergePOEnv-v0_0: RUNNING, 3 failures: /home/elizabeth/ray_results/stabilizing_open_network_merges/PPO_MergePOEnv-v0_0_2019-12-29_10-33-102tsagrr5/error_2019-12-29_10-34-31.txt

(pid=1375) WARNING:tensorflow:From /home/elizabeth/anaconda3/envs/MultiAgent/lib/python3.6/site-packages/tensorflow/python/compat/v2_compat.py:61: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. (pid=1375) Instructions for updating: (pid=1375) non-resource variables are not supported in the long term (pid=1375) 2019-12-29 10:34:35,080 WARNING ppo.py:143 -- FYI: By default, the value function will not share layers with the policy model ('vf_share_layers': False). 2019-12-29 10:46:34,384 ERROR worker.py:1714 -- The node with client ID 726b2cdb271f6b362b622698e881a5012e52a5ed has been marked dead because the monitor has missed too many heartbeats from it.

I am not sure whether it is beacuse I open too much parallel workers which my pc can't support? Or it is beacuse any other reasons.

flow-project / flow

Please add more tutorials for multiagent examples #804