search

autonomio / talos

Hyperparameter Experiments with TensorFlow and Keras
https://autonom.io
MIT License
1.62k stars 268 forks source link

[FEATURE REQUEST] Make iterating over epochs smarter #447

Closed Thomasillo closed 4 years ago

Thomasillo commented 4 years ago

1) I think Talos should add

Caching of runs or intermediate results in order to save time when just the number of epochs changes

2) Once implemented, I can see how this feature will

Save running time

3) I believe this feature is

4) Given the chance, I'd be happy to make a PR for this feature

Description:

The change of the number of epochs comes at minimal cost, when the intermediate model performance is saved. Eg when trainining a model at fixed values of hyperparameters with say 100 epochs, one could save the validation and training performance along with all other metrics for all numbers of epochs smaller than 100 at almost no additional cost.

A library like joblib might be very suitable to achieve this.

github-actions[bot] commented 4 years ago

Welcome to Talos community! Thanks so much for creating your first issue :)

mikkokotila commented 4 years ago

Cutting down time sounds always attractive :) Can you share more about your idea, because I did not understand how it would help us reduce experiment time.

mikkokotila commented 4 years ago

It's not at all clear how your proposal would make Scan() faster, because it seems to just suggest to not try different epochs as a parameter. I don't think one anyways needs to try different epochs as parameter, so closing here.

Feel free to comment and reopen if I have misunderstood.