ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

https://ray.io

Apache License 2.0

34.2k stars 5.81k forks source link

[tune] documentation/examples improvement thread #11707

Open richardliaw opened 4 years ago

richardliaw commented 4 years ago

Improving Tune examples

All examples should have

demonstrate how to retrieve best hyperparameter
verbose comments for code
set verbose=1 for better experience
Load the best model from checkpoint
- (Is there something I missed in the examples? or is the suggested workflow to just get the best hyper settings with ray and save them to a file which we train a totally new model with at a later time?)

Remove/deprioritize

trainable examples

Add:

More examples of using search algorithms (advanced usage)
links to tune-sklearn examples
A couple examples for continuing a hyperparameter search (reusing the search algorithm)
A couple examples for restoring/resuming a hyperparameter search

Also,

Have an example that uses create_* for schedulers and search algorithms

richardliaw commented 4 years ago

Tips and tricks page:

Start super simple -- with 1 trial and 1 GPU, with a 1 training_iteration step.
Using 'fail_fast'
Serialization issues → use diagnostic function
save/restore issues → use diagnostic function