search

facebook / Ax

Adaptive Experimentation Platform
https://ax.dev
MIT License
2.38k stars 312 forks source link

Can I use Ax for models being trained on multiple gpus? #718

Closed AmmaraRazzaq closed 3 years ago

AmmaraRazzaq commented 3 years ago

Can Ax be used to do hyperparameter tuning for a model that is being trained on multiple gpus using DistributedDataParallel in Pytorch?

danielcohenlive commented 3 years ago

Thanks for the question @AmmaraRazzaq. Yes, you just need to provide it a metric or evaluation function, depending on which API you're using, so that it can perform optimization. Where the model is being trained is not important to Ax, as it treats the thing it's optimizing as a blackbox.

danielcohenlive commented 3 years ago

Just in case I'm missing something, could you elaborate on what the challenges would be on either the parameter or the evaluation side of things?

lena-kashtelyan commented 3 years ago

Hi @AmmaraRazzaq, closing this issue as inactive and assuming your question was answered. Please reopen this issue and follow up if it was not!