facebookresearch / hydra

Hydra is a framework for elegantly configuring complex applications

https://hydra.cc

MIT License

8.84k stars 637 forks source link

[Feature Request] IBM LSF Launcher #1757

Open chuber1986 opened 3 years ago

chuber1986 commented 3 years ago

🚀 Feature Request

Support for IBM LSF cluster.

Motivation

IBM LSF is a widly used clusterization solution. I'd love to see hydra support LSF cluster similar to SLUM sluster.

Pitch

Analogous to the Hydra-SLURM launcher plugin, a Hydra-LSF launcher plugin would allow to deploy (multi-)runs on an LSF cluster. IBM provides an API similart to 'summit' which was used for the SLURM plugin ( https://github.com/IBMSpectrumComputing/lsf-python-api ).

omry commented 3 years ago

Hi @chuber1986, Currently we are not looking the expand the portfolio of launchers maintained by the Hydra team. You are welcome to create an LSF Launcher and publish it outside of the Hydra repo. Feel free to join the Hydra chat if you have questions about implementing such a launcher.

Here is an example launcher you can use as a base.

You can learn about plugin development here.

mmcdermott commented 4 weeks ago

While it seems like this should likely be closed as not planned in this repo, #2566 seems highly related.