search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

Interpreting .demo files. #2440

Closed rajatpaliwal closed 5 years ago

rajatpaliwal commented 5 years ago

Hi, In order to perform offline behavioral cloning we generate .demo files using demonstration recorder. Is there is a way to access the data stored in .demo file and make some inference outside Unity editor. Also, what is the format of the data collected( video, images, table, vector, text) in the .demo files.

vincentpierre commented 5 years ago

The .demo files are storing data in protobuf messages. These messages are the same used for communication between C# and Python. I am afraid extracting this data won't be easy, but you can look at the demo_loader if you really need to. If you are interested in Behavioral Cloning, you should check GAIL out.

rajatpaliwal commented 5 years ago

Hi @vincentpierre Since the protobuf messages are used for storing data in .demo files , is it possible to customize this data storage process through this guide provide in the document : https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Creating-Custom-Protobuf-Messages.md.

vincentpierre commented 5 years ago

I think this might be very difficult or even impossible. We would not be able to load those with the demo_loader if there are custom fields. This is not how the demonstrations were meant to be used.

rajatpaliwal commented 5 years ago

Hi, @vincentpierre Is there any way to customize or access the data sent to the neural net for training.

chriselion commented 5 years ago

Thank you for the discussion. We are closing this issue due to inactivity. Feel free to reopen it if you’d like to continue the discussion.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.