Is there any way to get the acc, loss etc for each epoch of the training?

[JohanMollevik]

Is there any way to get the acc, loss etc for each epoch of the training?

I was searching for that feature in the documentation but could not find it.

My use case for the feature is that I was running a rather long Scan and at the end I saw clear signs of over-training (acc much higher than val_acc). What I had hoped to do was look at the history of those two parameters to figure out where the over-training starts to later re run them.

Dos this feature exists and if so how? If not, should this be added to the feature roadmap?

[mikkokotila]

Hey, good to see you around :)

You can get it through:

scan_object.round_history

In there you'll get the keras history object for each processed permutation.

[renatobellotti]

That's good to know.

Is there a persistent way to do this? I'm running some Keras models on a cluster. The job needs many cores in order to speed up training. However, I don't want to have my job running for days just to have access to all histories...

Can the histories be logged somewhere?

[renatobellotti]

Of course, one can always save the histories in the end, but everything that can be done by the library means more productivity for the users of the library.

[mikkokotila]

I have been thinking about building an interactive real-time monitoring dashboard with dash-plotly, been experimenting with that with good success so far. Let's see if that happens soon, but definitely in there you would have access to history for all permutations during the experiment.

[mikkokotila]

Can you create a new issue for:

Can the histories be logged somewhere?

I'm closing here as it's resolved.