Support for AdaptDL

[pietrolesci]

Reporting from the idea-pool channel on slack, as discussed with @carmocca.

---

Hi there,

On the way to solve a OOM problem with dynamic batch sizes based on sequence length, I have just discovered AdaptDL. Might be an interesting library to support.

Some core features offered by AdaptDL are:

• Elastically schedule distributed DL training jobs in shared clusters.
• Cost-aware resource auto-scaling in cloud computing environments (e.g. AWS).
• Automatic batch size and learning rate scaling for distributed training.

[carmocca]

Hi! Thanks for the request.

Note however that we would only be interested in integrating the training tricks they provide such as batch size and learning rate scaling features.

We already support easy and elastic scaling of cloud jobs via our Lightning.ai platform. You can find ready-to-use apps such as the PL app to launch your PL scripts, or the demo app to get started.