Open dxoigmn opened 1 year ago
Right now LitModular does not support ChainedScheduler. This is because a ChainedScheduler does not take an optimizer, but rather a list of schedulers: https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ChainedScheduler.html
LitModular
ChainedScheduler
LitModular assumes lr_scheduler is a callable that receives an Optimizer: https://github.com/IntelLabs/MART/blob/41ad18e3aa08b420267772bd197e8c610389d2cc/mart/models/modular.py#L122
lr_scheduler
My sense is the best way to fix this is to allow lr_scheduler to be a list and we can create the underlying ChainedScheduler within LitModular.
Right now
LitModulardoes not supportChainedScheduler. This is because aChainedSchedulerdoes not take an optimizer, but rather a list of schedulers: https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.ChainedScheduler.htmlLitModularassumeslr_scheduleris a callable that receives an Optimizer: https://github.com/IntelLabs/MART/blob/41ad18e3aa08b420267772bd197e8c610389d2cc/mart/models/modular.py#L122My sense is the best way to fix this is to allow
lr_schedulerto be a list and we can create the underlyingChainedSchedulerwithinLitModular.