search

ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.07k stars 5.6k forks source link

[Tune] support for FIRE PBT #24137

Open tfriedel opened 2 years ago

tfriedel commented 2 years ago

Description

You already support PBT (population based training). One of the shortcomings of PBT is that it tends to favor short-term improvements. To address this issue, a DeepMind research team has proposed Faster Improvement Rate PBT (FIRE PBT), a novel approach that outperforms PBT methods and matches the performance of networks trained with traditional manual hyperparameter tuning on the ImageNet benchmark. The method is described in this paper.

I think adding this method to Ray Tune would be very useful. Do you think you could add it?

xwjiang2010 commented 2 years ago

Yes, this is on our backlog. cc @krfricke @Yard1 we should think about prioritizing it.

stale[bot] commented 2 years ago

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

You can always ask for help on our discussion forum or Ray's public slack channel.

stale[bot] commented 1 year ago

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

You can always ask for help on our discussion forum or Ray's public slack channel.

AwesomeLemon commented 1 year ago

Hi, is there any estimate on when this will be implemented?