search

ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.67k stars 5.72k forks source link

[Feature Request] [Tune] Add a special 'evaluation-step' flag to avoid unnecessary lengthy evaluations #11725

Open erezinman opened 4 years ago

erezinman commented 4 years ago

I have a suggestion for a small change in Ray-Tune's Trainable - add the flag is_evaluation_step as a member to allow for better control over when evaluate the model. It could speed-up performance greatly in tuning algorithms where inference takes a long time (such as tuning the very popular variants of 1NN-DTW for time-series, or similar); or when the tuning-by-metric takes a lot of time to calculate.

Originally posted by @erezinman in https://github.com/ray-project/ray/discussions/11723

Also, discussed in this discussion and added as an issue per @richardliaw's request.

erezinman commented 4 years ago

Can't delete the discussion. I opened it prior to your request to open it as an issue, so I linked to it