search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

Using --load rewrite the entire csv status summary making it useless #3326

Closed caioc2 closed 4 years ago

caioc2 commented 4 years ago

Describe the bug Using --load rewrite the entire csv status summary making it useless. When using --load option it should either create another .csv with the newer data (much like tensorflow does) or append to the exiting .csv file. If it doesn't, all stats saved before --load are lost which makes it pretty useless for generating any graphics or statistics.

To Reproduce Steps to reproduce the behavior:

  1. Run any environment for a while and cancel it
  2. Start it again with --load option
  3. the summaries/ *.csv will be rewritten containing only the steps after the --load.

Console logs / stack traces No logs or stack-trace

Environment (please complete the following information):

My currently work around

anupam-142857 commented 4 years ago

This is not a bug, it is the intended behavior.

caioc2 commented 4 years ago

Then I kindly ask to review its design.

Is it any usefull to have a csv that is not complete and overrides the older when --load clearly is a continuation of a training and not really a new one?

For any purposes of ploting a graph or garthering statistics, having half a the points makes no sense.

EgonGoodman commented 4 years ago

hey caioc2,

I have exactly the same problem. There is no use for --load in my opinion. I thought it will continue my session, but its starts a new one. If I would like to start a new session I would do it without --load. Is there another option to continue the training?

Thank you for your help.

caioc2 commented 4 years ago

Hi @TanteVonDante,

Here I mean the ".csv is unusable after a --load because it is overwritten", yet it works for the model, note that the model starts from where it stopped, the environment starts fresh.

You can get the data from the tensorflow generated summary (opening it and converting to csv), but them, there is no point in having it to save a unusable .csv just to bloat the disk.

EgonGoodman commented 4 years ago

maybe I do something wrong, but when I use --load the model acts like it never trained before.

I wanted to use it instead of curriculum learning. I wanted to train like 50.000 steps and change the environment afterwards. So the training will get harder step by step. Is this possible?

dlindmark commented 4 years ago

maybe I do something wrong, but when I use --load the model acts like it never trained before.

I wanted to use it instead of curriculum learning. I wanted to train like 50.000 steps and change the environment afterwards. So the training will get harder step by step. Is this possible?

What mlagents version are you using? 0.13.0 had a bug which made the step count reset in the tensorboard summaries. The model continued training as intended though (I think). Fixed in 0.13.1.

EgonGoodman commented 4 years ago

Where can I check the version? I just started last december with ml and coding, so I'm a total beginner.

anupam-142857 commented 4 years ago

@TanteVonDante to verify the version numbers please see this link https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md#warning-warning-warning

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.