Smarter selection of Batch size during hyper-parameter selection and training

[Anindyadeep]

Prompt2Model currently has a static way of defining batch size. The user has to tweak it into the code to either train models faster. Also referencing this issue #315, the batch size is also a hyper-parameter that is important. The way in general this hyperparameter search works is, that we need to provide a list of common batch sizes and during optimization, the correct batch size combination is been selected.

However, for certain device and model combinations, selecting batch size >= 4 might create OOM issues. So, before selection batch size for training or hyperparameter optimization, we need to check for the upper bound automatically such that OOM issues are also not raised and the full resource utilization is also there.

[zhaochenyang20]

Is it related to this PR: https://github.com/neulab/prompt2model/pull/315

[Anindyadeep]

Yes, as we are more into automated hyper-parameter selection, so the batch size should also need to be automated. However, I am keeping that to Todo and have kept a separate issue to solve.