Include python packages from modules

[ccoulombe]

On the available python wheel page, it would be really nice to include python packages included in module. Some examples are:

• pyarrow : arrow module
• mpi4py : mpi4py module
• petsc4py: petsc module

One option could be to maintain an index as done with Mii. Or parse LMOD output?

[mboisson]

The output of spider can be used for that, in the json output, the provides field details which extensions are provided by the module:

    "Core/igraph": {
        "/cvmfs/soft.computecanada.ca/easybuild/modules/2020/avx2/Core/igraph/0.8.2.lua": {
            "pV": "000000000.000000008.000000002.*zfinal",
            "Description": "igraph is a collection of network analysis tools with the emphasis on \nefficiency, portability and ease of use. igraph is open source and free. igraph can be \nprogrammed in R, Python and C/C++.",
            "URL": "http://igraph.org",
            "pathA": {
                "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/igraph/0.8.2/bin": 1
            },
            "whatis": [
                "Description: igraph is a collection of network analysis tools with the emphasis on \nefficiency, portability and ease of use. igraph is open source and free. igraph can be \nprogrammed in R, Python and C/C++.",
                "Homepage: http://igraph.org",
                "URL: http://igraph.org",
                "Compatible modules: python/3.6, python/3.7, python/3.8",
                "Extensions: python-igraph-0.8.2"
            ],
            "wV": "000000000.000000008.000000002.*zfinal",
            "hidden": false,
            "Version": "0.8.2",
            "fullName": "Core/igraph/0.8.2",
            "help": "\nDescription\n===========\nigraph is a collection of network analysis tools with the emphasis on \nefficiency, portability and ease of use. igraph is open source and free. igraph can be \nprogrammed in R, Python and C/C++.\n\n\nMore information\n================\n - Homepage: http://igraph.org\n\n\nCompatible modules\n==================\nThis module is compatible with the following modules, one of each line is required:\n* python/3.6, python/3.7, python/3.8\n\n\nIncluded extensions\n===================\npython-igraph-0.8.2\n",
            "provides": [
                "python-igraph/0.8.2"
            ],
            "lpathA": {
                "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/igraph/0.8.2/lib": 1
            }
        }
    },

[ccoulombe]

I does also includes Perl extensions right? In that case, it would not work out of the box

[mboisson]

It includes any extension. Could filter based on the mention of python in the description.

Although, listing R or Perl extensions would also be useful.

[ccoulombe]

This issue is basically a duplicate and will resolve itself with : https://github.com/ComputeCanada/avail_wheels/issues/3 Once avail_wheels list wheels from modules, the Available Python Wheel page will include them.