koomie
commented
4 years ago Thank you for the submission. The TSC has recommended acceptance of this component into a future release and a new tracker has been created in openhpc/ohpc#1188 for completion of the integration work
Software Name
magpie
Public URL
https://github.com/LLNL/magpie
Technical Overview
Magpie is a collection of scripts for running big data and AI frameworks in an HPC environment. The scripts facilitate the submission of these jobs through an HPC scheduler on typical HPC cluster architecture. Magpie supports the major big data frameworks, as well as TensorFlow and Ray. For example, submitting a Tensorflow/Horovod workload over Apache Spark, using SLURM as the job scheduler.
Magpie installation is simple. It's a directory of scripts that must be made available on all cluster nodes. The goal here is to make it easier to for users to get and update to the latest release; plus, the OpenHPC pub subdirectory is a good location to host the scripts.
Latest stable version number
2.3
Open-source license type
GPL 2.0
Relationship to component?
If other, please describe: Used as component in redistributed reference designs.
Build system
If other, please describe: Tarball of scripts.
Does the current build system support staged path installations? For example:
make install DESTIR=/tmp/foo(or equivalent)Does component run in user space or are administrative credentials required?
Does component require post-installation configuration.
If yes, please describe briefly: It may need to be configured as different data/ML environments are implemented. I expect this to outside of the magpie RPM scope, with the exception of any configuration required for integration into SLURM.
If component is selected, are you willing and able to collaborate with OpenHPC maintainers during the integration process?
Does the component include test collateral (e.g. regression/verification tests) in the publicly shipped source?
If yes, please briefly describe the intent and location of the tests. Included in a testsuite subdrectory. Includes various tests for different data frameworks.
Does the component have additional software dependencies (beyond compilers/MPI) that are not part of standard Linux distributions?
If yes, please list the dependencies and associated licenses.
Does the component include online or installable documentation?
If available online, please provide URL.
[Optional]: Would you like to receive additional review feedback by email?
- [ ] yes - [X] no