ARM-software / mango

Parallel Hyperparameter Tuning in Python
Apache License 2.0
335 stars 40 forks source link

`n_iterations` #98

Open tihom opened 1 year ago

tihom commented 1 year ago

@sandeep-iitr there have been few issues about the meaning of n_iterations in parallel mode. Intuitively (since all other tuning libraries have this behavior) n_iterations should be same as number of objective function evaluation but Mango does n_iterations x batch_size.

I propose that we deprecate n_iterations and add a new parameter: n_samples Number of function evaluation would always be == n_samples in serial and parallel cases. In parallel case the number of batches is calculated as n_samples / batch_size.

We would add a deprecation warning to n_iterations parameter to warn users about the change and update all docs and examples to use n_samples. This way it won't break existing usage.

I can make all the changes needed.

sandeep-iitr commented 1 year ago

Hi Mohit, Thanks for raising. I agree with your observations. More recently, we have seen many users asking for clarifications on the total number of samples vs the number of iterations parameter.