search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.93k stars 4.14k forks source link

Import keras model into Unity3D environment #3096

Closed IRiViI closed 4 years ago

IRiViI commented 4 years ago

The Unity ml agents environt is amazing and the PPO agent as well. However I would like to have more flexibility on the model and the training paramters. Additionally I would like to import the trained model in the Unity3D editor.

My model is running happily in jupyter but I can't import it into unity, which makes me sad. I did manage to convert it into a .nn file (with the help of internet). I also tried to understand the policy, model and training code but I think that would be a bit of excessive for what I want I would think.

Sooo my question is, which kind of things do I have to add to my Keras model in order to make it importable into the Unity3D editor?

The Error I get:

NullReferenceException: Object reference not set to an instance of an object MLAgents.InferenceBrain.BarracudaModelParamLoader.CheckModel (Barracuda.Model model, MLAgents.BrainParameters brainParameters) (at Assets/ML-Agents/Scripts/InferenceBrain/BarracudaModelParamLoader.cs:117)

My Code: `

The model

input_obs_shape=(84,84,3) input_obs_layer = Input(input_obs_shape, name="visual_observation_0") obs_layer = Reshape((84843,))(input_obs_layer) layers = Dense(10, activation="relu")(obs_layer) layers = Dense(4, activation="sigmoid", name="action_probs/action_probs")(layers) model = Model(input_obs_layer , layers) model.compile(loss="mse", optimizer="adam")

Testing the model

brain_infos = env.reset(train_mode=False) default_brain = env.external_brain_names[0] brain_info = brain_infos[default_brain]

prediction = model.predict(np.array(brain_info.visual_observations[0] ))

Freezing the model (using https://github.com/tensorflow/tensorflow/issues/32241)

frozen_graph = freeze_session(K.get_session(), output_names=[out.op.name for out in model.outputs])

Exporting the model

tf.train.write_graph(frozen_graph, "pb_directory", "my_model.pb", as_text=False)

model_path = "temptemp" tf2bc.convert('pb_directory/my_model.pb', model_path + ".nn") `

anupam-142857 commented 4 years ago

HI @IRiViI at present we do not support this functionality. However, in theory it's possible to import a Keras model into Unity. The keras model needs to be saved into onnx format and then convert it into barracuda using the onnx to barracuda converter and then use it in Unity. This sounds simple in theory but the reality might be different.

IRiViI commented 4 years ago

HI @IRiViI at present we do not support this functionality. However, in theory it's possible to import a Keras model into Unity. The keras model needs to be saved into onnx format and then convert it into barracuda using the onnx to barracuda converter and then use it in Unity. This sounds simple in theory but the reality might be different.

Thank you for your reponse and sorry for my late reply. Too bad that it's not that simple using the current tools. I will probably try to check out what I can achieve using python code and the ml agents env tools. The reason why I wanted to intergrate keras into unity is because I want to move freely trough my environment during training. Hopefully there will be another way. Thanks again for your reply

chriselion commented 4 years ago

Hi @IRiViI With respect to the particular error that you're hitting - we look for a few particular constants in the barracuda file at load time: https://github.com/Unity-Technologies/ml-agents/blob/0.12.1/UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/BarracudaModelParamLoader.cs#L141-L145

In the normal workflow, these constants are added to the tensorflow model e.g. here: https://github.com/Unity-Technologies/ml-agents/blob/0.12.1/ml-agents/mlagents/trainers/models.py#L64-L69

You'll need to make sure these constants are set in your keras model before converting to barracuda.

Unfortunately, the conventions for how the python code sets up the model and how the barracuda importer looks for tensors are pretty tightly coupled, so you might hit similar problems even if you get past this one. The strings are all in https://github.com/Unity-Technologies/ml-agents/blob/0.12.1/UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TensorNames.cs.

We'll eventually use ONNX instead of .nn files, but it's not ready yet; we need a few fixes from the barracuda team to make the import process easier.

Note that if you just want to use barracuda for inference, you can do that without ML-Agents (and ONNX import should work fine). The documention for that is with the package: https://docs.unity3d.com/Packages/com.unity.barracuda@0.3/manual/index.html

IRiViI commented 4 years ago

Hi @IRiViI With respect to the particular error that you're hitting - we look for a few particular constants in the barracuda file at load time: https://github.com/Unity-Technologies/ml-agents/blob/0.12.1/UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/BarracudaModelParamLoader.cs#L141-L145

In the normal workflow, these constants are added to the tensorflow model e.g. here: https://github.com/Unity-Technologies/ml-agents/blob/0.12.1/ml-agents/mlagents/trainers/models.py#L64-L69

You'll need to make sure these constants are set in your keras model before converting to barracuda.

Unfortunately, the conventions for how the python code sets up the model and how the barracuda importer looks for tensors are pretty tightly coupled, so you might hit similar problems even if you get past this one. The strings are all in https://github.com/Unity-Technologies/ml-agents/blob/0.12.1/UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TensorNames.cs.

We'll eventually use ONNX instead of .nn files, but it's not ready yet; we need a few fixes from the barracuda team to make the import process easier.

Note that if you just want to use barracuda for inference, you can do that without ML-Agents (and ONNX import should work fine). The documention for that is with the package: https://docs.unity3d.com/Packages/com.unity.barracuda@0.3/manual/index.html

Thanks a lot for the references! I already found a few of them but not the list of all string values! Do you have an indication when the new version will be ready? If it will be a month or two it might be worth waiting for me =).

chriselion commented 4 years ago

hi @IRiViI, I don't have an estimate from the barracuda team on when ONNX importer changes will be ready. We're also blocked on tf2onnx supporting tensorflow 2.0 (see https://github.com/onnx/tensorflow-onnx/issues/691),

Note that the tight coupling that I mentioned between the tensor names will remain for the immediate future.

IRiViI commented 4 years ago

hi @IRiViI, I don't have an estimate from the barracuda team on when ONNX importer changes will be ready. We're also blocked on tf2onnx supporting tensorflow 2.0 (see onnx/tensorflow-onnx#691),

Note that the tight coupling that I mentioned between the tensor names will remain for the immediate future.

Thank you for the information. The tight coupling seems reasonable. I think I will just play within python for now and try to tackle this problem lateron when (maybe) some changes to ml agents are ready.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.