search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

ValueError: Cannot feed value of shape #502

Closed Lakrix closed 6 years ago

Lakrix commented 6 years ago

I'm trying to run teacher/student mode. I have followed the guide on how to set up imitation learning. When I try to run learn.py I get the following error.

loss, _ = self.sess.run([self.model.loss, self.model.update], feed_dict=feed_dict) File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run run_metadata_ptr) File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1096, in _run % (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (4, 4) for Tensor 'teacher_action:0', which has shape '(?,)'

If I switch my teacher brain to external then Im able train the brain. Im not able to have my teacher brain as player input together with my student as external input.

vincentpierre commented 6 years ago

I do not run in to this error. Could you try cloning again the repo on the master branch and try again. The banana imitation scene should be ready for use. I have never encountered this error but my best guess is that the teacher brain and student brain may have different brain parameters (continuous vs discrete vector action space).

Lakrix commented 6 years ago

This only happens when I use a discrete action space. When Im using a continous action space everything works fine.

'PlayerAcademy' started successfully! Unity Academy name: PlayerAcademy Number of Brains: 2 Number of External Brains : 1 Lesson number : 0 Reset Parameters :

Unity brain name: TeacherBrain Number of Visual Observations (per agent): 0 Vector Observation space type: continuous Vector Observation space size (per agent): 10 Number of stacked Vector Observation: 1 Vector Action space type: discrete Vector Action space size (per agent): 4 Vector Action descriptions: , , , Unity brain name: StudentBrain Number of Visual Observations (per agent): 0 Vector Observation space type: continuous Vector Observation space size (per agent): 10 Number of stacked Vector Observation: 1 Vector Action space type: discrete Vector Action space size (per agent): 4 Vector Action descriptions: , , ,

vincentpierre commented 6 years ago

Did you make sure the teacher and student brains had the same parameters ? (same action space type ?)

Lakrix commented 6 years ago

Yes they have the same type. If the action space is continuous it works but not for discrete.

vincentpierre commented 6 years ago

I was able to reproduce your error. I made some fixes to the trainer, changes are in this Pull Request. Let us know if it works.

Lakrix commented 6 years ago

Yes it works to run the it now. But after max steps I get the following error.

ddTraceback (most recent call last): File "learn.py", line 62, in tc.start_learning() File "/Documents/UnityMlAgentsTest/ml-agents0.3/ml-agents/python/unitytrainers/trainer_controller.py", line 280, in start_learning self._export_graph() File "/Documents/UnityMlAgentsTest/ml-agents0.3/ml-agents/python/unitytrainers/trainer_controller.py", line 145, in _export_graph restore_op_name="save/restore_all", filename_tensor_name="save/Const:0") File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/tools/freeze_graph.py", line 239, in freeze_graph input_meta_graph_def, input_saved_model_dir, saved_model_tags.split(",")) File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/tools/freeze_graph.py", line 150, in freeze_graph_with_def_protos variable_names_blacklist=variable_names_blacklist) File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py", line 208, in convert_variables_to_constants inference_graph = extract_sub_graph(input_graph_def, output_node_names) File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py", line 145, in extract_sub_graph assert d in name_to_node_map, "%s is not in graph" % d AssertionError: action is not in graph

vincentpierre commented 6 years ago

I think version 0.3.0b resolved that issue. Let us know if you are still facing this issue.

xiaomaogy commented 6 years ago

I am closing this issue due to inactivity. Feel free to reopen it if it is still not fixed in the 0.3.0b.

leobol96 commented 5 years ago

Hello, I think that i have the same issue using learning by demostration. Brains have the same parameters.

This is the error message:

INFO:mlagents.envs:Hyperparameters for the OfflineBCTrainer of brain RightBrain: trainer: offline_bc batch_size: 256 summary_freq: 1000 max_steps: 5.0e6 batches_per_epoch: 10 use_recurrent: True hidden_units: 256 learning_rate: 0.0003 num_layers: 3 sequence_length: 32 memory_size: 256 demo_path: nodescription_1.demo summary_path: ./summaries/ppo_RightBrain model_path: ./models/ppo-0/RightBrain keep_checkpoints: 5 Traceback (most recent call last): File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\Leonardo\Anaconda3\envs\ml-agents\Scripts\mlagents-learn.exe__main__.py", line 9, in File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\learn.py", line 337, in main run_training(0, run_seed, options, Queue()) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\learn.py", line 132, in run_training tc.start_learning(env) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\trainer_controller.py", line 202, in start_learning n_steps = self.advance(env_manager) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\envs\timers.py", line 261, in wrapped return func(*args, **kwargs) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\trainer_controller.py", line 292, in advance trainer.update_policy() File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\bc\trainer.py", line 138, in update_policy run_out = self.policy.update(mini_batch, self.n_sequences) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\bc\policy.py", line 95, in update run_out = self._execute_model(feed_dict, self.update_dict) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\mlagents\trainers\tf_policy.py", line 148, in _execute_model network_out = self.sess.run(list(out_dict.values()), feed_dict=feed_dict) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\tensorflow\python\client\session.py", line 905, in run run_metadata_ptr) File "c:\users\leonardo\anaconda3\envs\ml-agents\lib\site-packages\tensorflow\python\client\session.py", line 1116, in _run str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (8, 5) for Tensor 'teacher_action:0', which has shape '(?, 4)'

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.