Is it possible to "update" a model?

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.

https://unity.com/products/machine-learning-agents

Other

17.19k stars 4.16k forks source link

Is it possible to "update" a model? #3162

Closed fredleefarr closed 4 years ago

fredleefarr commented 4 years ago

If I'm training a new model, I can generate one with the following example settings:

mlagents-learn config/mybehavior_config.yaml --train

I can then drag the generated "testbehavior.nn" into the "Model" section and can see the results of learning by clicking play in the Unity editor.

If I keep the "TestBehavior" in the Model:

And then re-run the training command again:

mlagents-learn config/mybehavior_config.yaml --train

Does the model get updated? I tried testing this and it didn't appear to be "adding" to the model. Is it just a case that models are created once during training, and once done they cannot be changed?

Rodnyy commented 4 years ago

Hi, as far as I know, it is possible to continue training the model but not the way you tried. You simply have to run the training command again and add "--load" at the end also do not forget to increase the max steps in the configuration if you already hit it. during previous learning. If you still have any questions regarding this topic I am sure I have seen more detailed answers about this topic here.

fredleefarr commented 4 years ago

Many thanks @Rodnyy, will look at the --load option.

surfnerd commented 4 years ago

Hi @fredleefarr, The model that you highlighted in the editor was added there after training. If you'd like to update your NN model in the editor, look for the end of the training log to see where it is. You can then drag that file into the editor and use it for inference if you'd like.

fredleefarr commented 4 years ago

Thanks @surfnerd - I realise that I need to be careful about using the correct terminology with ML. By "update" I meant "continue to train the model" after initial training. I'll investigate the --load option @Rodnyy mentioned.

surfnerd commented 4 years ago

Ah, sorry for my misunderstanding. I hope you were able to get it to work.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.