LeoGrin / tabular-benchmark

448 stars 59 forks source link

wandb number of random search iterations - default value #12

Closed lumliolum closed 1 year ago

lumliolum commented 1 year ago

Hello,

While creating sweep for a model, hyperparameter method in sweep_config is either random or grid. The code for this is given here (see line 41)

https://github.com/LeoGrin/tabular-benchmark/blob/3ea416e5e20aa254dab1771ad5587f37bc88d4d6/launch_config/utils.py#L36-L51

If the method is random, then what is number of random iterations that wandb will use ?

I checked the wandb documentation here about sweep strategy : https://docs.wandb.ai/guides/sweeps/define-sweep-configuration#configuration-keys

but couldn't find the default value of random search iterations

LeoGrin commented 1 year ago

If the method is random, wandb will keep on going until you tell it to stop. In the updated version of the code, I shared some code (https://github.com/LeoGrin/tabular-benchmark/blob/refactoring/src/launch_benchmarks/launch_on_cluster.py) to automatically stop wandb sweeps when enough runs have been finished.