When I set the number of agents to 256, I got an error.
But setting the number of agents to a smaller number will work.
How do I solve this problem, and what is the maximum number of agents?
# Error logs:
tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and:
`tf.data.TFRecordDataset(path)`
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/data_generators/gym_env.py:541: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.
Instructions for updating:
Use eager execution and:
`tf.data.TFRecordDataset(path)`
INFO:tensorflow:Initial training of the policy in real environment.
INFO:tensorflow:Initial training of the policy in real environment.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
INFO:tensorflow:Applying wrapper <class 'tensor2tensor.rl.envs.tf_atari_wrappers.StackWrapper'>({'history': 4}) to env PyFuncEnv(T2TGymEnv(<TimeLimitMaxAndSkip<<AtariEnv<PongNoFrameskip-v4>>>>)).
INFO:tensorflow:Applying wrapper <class 'tensor2tensor.rl.envs.tf_atari_wrappers.StackWrapper'>({'history': 4}) to env PyFuncEnv(T2TGymEnv(<TimeLimitMaxAndSkip<<AtariEnv<PongNoFrameskip-v4>>>>)).
INFO:tensorflow:Applying wrapper <class 'tensor2tensor.rl.ppo_learner._MemoryWrapper'>({}) to env StackWrapper(PyFuncEnv(T2TGymEnv(<TimeLimitMaxAndSkip<<AtariEnv<PongNoFrameskip-v4>>>>))).
INFO:tensorflow:Applying wrapper <class 'tensor2tensor.rl.ppo_learner._MemoryWrapper'>({}) to env StackWrapper(PyFuncEnv(T2TGymEnv(<TimeLimitMaxAndSkip<<AtariEnv<PongNoFrameskip-v4>>>>))).
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/envs/py_func_batch_env.py:122: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
tf.py_function, which takes a python function which manipulates tf eager
tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means `tf.py_function`s can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/envs/py_func_batch_env.py:122: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
tf.py_function, which takes a python function which manipulates tf eager
tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means `tf.py_function`s can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
INFO:tensorflow:Using DummyPolicyProblem for the policy.
INFO:tensorflow:Using DummyPolicyProblem for the policy.
INFO:tensorflow:Setting T2TModel mode to 'train'
INFO:tensorflow:Setting T2TModel mode to 'train'
INFO:tensorflow:Using variable initializer: orthogonal
INFO:tensorflow:Using variable initializer: orthogonal
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/utils/t2t_model.py:1358: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/utils/t2t_model.py:1358: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
INFO:tensorflow:Transforming feature 'input_action' with symbol_modality_6_64.bottom
INFO:tensorflow:Transforming feature 'input_action' with symbol_modality_6_64.bottom
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/function.py:1007: calling Graph.create_op (from tensorflow.python.framework.ops) with compute_shapes is deprecated and will be removed in a future version.
Instructions for updating:
Shapes are always computed; don't use the compute_shapes as it has no effect.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/function.py:1007: calling Graph.create_op (from tensorflow.python.framework.ops) with compute_shapes is deprecated and will be removed in a future version.
Instructions for updating:
Shapes are always computed; don't use the compute_shapes as it has no effect.
INFO:tensorflow:Transforming feature 'input_reward' with symbol_modality_3_64.bottom
INFO:tensorflow:Transforming feature 'input_reward' with symbol_modality_3_64.bottom
INFO:tensorflow:Transforming feature 'inputs' with video_modality.bottom
INFO:tensorflow:Transforming feature 'inputs' with video_modality.bottom
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/layers/common_layers.py:277: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/layers/common_layers.py:277: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
INFO:tensorflow:Transforming feature 'target_action' with symbol_modality_6_64.targets_bottom
INFO:tensorflow:Transforming feature 'target_action' with symbol_modality_6_64.targets_bottom
INFO:tensorflow:Transforming feature 'target_policy' with identity_modality.targets_bottom
INFO:tensorflow:Transforming feature 'target_policy' with identity_modality.targets_bottom
INFO:tensorflow:Transforming feature 'target_reward' with symbol_modality_3_64.targets_bottom
INFO:tensorflow:Transforming feature 'target_reward' with symbol_modality_3_64.targets_bottom
INFO:tensorflow:Transforming feature 'target_value' with identity_modality.targets_bottom
INFO:tensorflow:Transforming feature 'target_value' with identity_modality.targets_bottom
INFO:tensorflow:Transforming feature 'targets' with video_modality.targets_bottom
INFO:tensorflow:Transforming feature 'targets' with video_modality.targets_bottom
INFO:tensorflow:Building model body
INFO:tensorflow:Building model body
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:598: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv2d instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:598: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv2d instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:602: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:602: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:603: dropout (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dropout instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:603: dropout (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dropout instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:604: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/models/research/rl.py:604: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
INFO:tensorflow:Transforming body output with identity_modality.top
INFO:tensorflow:Transforming body output with identity_modality.top
INFO:tensorflow:Transforming body output with identity_modality.top
INFO:tensorflow:Transforming body output with identity_modality.top
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/layers/common_layers.py:2887: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.random.categorical instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/layers/common_layers.py:2887: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.random.categorical instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py:479: Print (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2018-08-20.
Instructions for updating:
Use tf.print instead of tf.Print. Note that tf.print returns a no-output operator that directly prints the output. Outside of defuns or eager mode, this operator will not be executed unless it is directly specified in session.run or used as a control dependency for other operators. This is only a concern in graph mode. Below is an example of how to ensure tf.print executes in graph mode:
```python
sess = tf.Session()
with sess.as_default():
tensor = tf.range(10)
print_op = tf.print(tensor)
with tf.control_dependencies([print_op]):
out = tf.add(tensor, tensor)
sess.run(out)
Additionally, to use tf.print in python 2.7, users must make sure to import
the following:
from __future__ import print_function
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py:479: Print (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2018-08-20.
Instructions for updating:
Use tf.print instead of tf.Print. Note that tf.print returns a no-output operator that directly prints the output. Outside of defuns or eager mode, this operator will not be executed unless it is directly specified in session.run or used as a control dependency for other operators. This is only a concern in graph mode. Below is an example of how to ensure tf.print executes in graph mode:
sess = tf.Session()
with sess.as_default():
tensor = tf.range(10)
print_op = tf.print(tensor)
with tf.control_dependencies([print_op]):
out = tf.add(tensor, tensor)
sess.run(out)
Additionally, to use tf.print in python 2.7, users must make sure to import
the following:
from __future__ import print_function
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 387, in
tf.app.run()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 382, in main
training_loop(hp, FLAGS.output_dir)
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 291, in training_loop
train_agent_real_env(env, learner, hparams, epoch)
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 192, in train_agent_real_env
num_env_steps=num_env_steps,
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py", line 85, in train
force_beginning_resets=simulated))
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py", line 166, in _define_train
**collect_kwargs))
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py", line 489, in _define_collect
"The rollout of ppo_hparams.epoch_length will be distributed amongst"
AssertionError: The rollout of ppo_hparams.epoch_length will be distributed amongsteffective_num_agents of agents
Description
When I set the number of agents to 256, I got an error. But setting the number of agents to a smaller number will work. How do I solve this problem, and what is the maximum number of agents?
My run command:
Additionally, to use tf.print in python 2.7, users must make sure to import the following:
from __future__ import print_functionWARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py:479: Print (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2018-08-20. Instructions for updating: Use tf.print instead of tf.Print. Note that tf.print returns a no-output operator that directly prints the output. Outside of defuns or eager mode, this operator will not be executed unless it is directly specified in session.run or used as a control dependency for other operators. This is only a concern in graph mode. Below is an example of how to ensure tf.print executes in graph mode:
Additionally, to use tf.print in python 2.7, users must make sure to import the following:
from __future__ import print_functionTraceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 387, in
tf.app.run()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 382, in main
training_loop(hp, FLAGS.output_dir)
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 291, in training_loop
train_agent_real_env(env, learner, hparams, epoch)
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/trainer_model_based.py", line 192, in train_agent_real_env
num_env_steps=num_env_steps,
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py", line 85, in train
force_beginning_resets=simulated))
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py", line 166, in _define_train
**collect_kwargs))
File "/usr/local/lib/python3.6/dist-packages/tensor2tensor/rl/ppo_learner.py", line 489, in _define_collect
"The rollout of ppo_hparams.epoch_length will be distributed amongst"
AssertionError: The rollout of ppo_hparams.epoch_length will be distributed amongsteffective_num_agents of agents