search

easybuilders / easybuild

EasyBuild - building software with ease
http://easybuild.io
GNU General Public License v2.0
461 stars 143 forks source link

add default moduleclass for AI/ML software? #815

Closed migueldiascosta closed 1 year ago

migueldiascosta commented 2 years ago

currently, AI/ML software is categorized rather loosely: scikit-learn is data, pytorch is devel, tensorflow is lib, keras is math...

there's nothing about AI/ML in the list of default moduleclasses, https://github.com/easybuilders/easybuild-framework/blob/454fe5e024b443b189f8e0f9f8727ed54c28ff56/easybuild/tools/config.py#L400-L424 (but it's not that they are not specific, there's even quantum :) )

should we do something about this?

boegel commented 2 years ago

Yeah, the time is right to add a new default module class and start using it. Do we want to go with ai, or ai_ml, etc.?

I guess ai is broad enough to also cover machine learning?

migueldiascosta commented 2 years ago

Another question is if we would reclassify existing easyconfigs or just new ones

boegel commented 2 years ago

I think re-classifying makes sense. PRs like that are trivial enough to merge without requiring test reports for all easyconfigs we touch (which can be brutally expensive for software like TensorFlow and PyTorch...).

The downside of that would be that trying to use an easyconfig that has been updated to use ai as module class would result in the following unless a sufficiently recent version of EasyBuild framework (develop branch or the to-be-released EasyBuild v4.6.1) is used:

ERROR: Failed to process easyconfig /tmp/example.eb: moduleclass provided 'ai' is not valid: ['base', 'astro', 'bio', 'cae', 'chem', 'compiler', 'data', 'debugger', 'devel', 'geo', 'ide', 'lang', 'lib', 'math', 'mpi', 'numlib', 'perf', 'quantum', 'phys', 'system', 'toolchain', 'tools', 'vis']