Hail (Python)

[la-mendicino]

Hi CCR team, I am requesting the library Hail be installed (https://hail.is/docs/0.2/install/linux.html). Thanks!

[dsajdak]

Hi @la-mendicino! There is no Easybuild recipe for this. I would recommend trying this out in a virtual environment. We have documentation on that here. When I tried this I loaded some of our existing modules to get access to some of the python libraries this software requires. Then I created a virtual environment, activated it, and used pip to install hail. Like this:

 module load gcc/11.2.0 openmpi/4.1.1 scipy-bundle/2021.10 plotly.py/5.12.0 pyyaml/6.0 pillow
 python3 -m venv /projects/academic/ccradmintest/dsajdaktest/hail
 source /projects/academic/ccradmintest/dsajdaktest/hail/bin/activate
(hail) dsajdaktest@login1:~$ pip install hail

It goes through the process of verifying dependencies and then attempting to install whatever else it still needs. You will see things like this:

  Attempting uninstall: numpy
    Found existing installation: numpy 1.21.3
    Not uninstalling numpy at /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/MPI/gcc/11.2.0/openmpi/4.1.1/scipy-bundle/2021.10/lib/python3.9/site-packages, outside environment /projects/academic/ccradmintest/dsajdaktest/hail
    Can't uninstall 'numpy'. No files were found to uninstall.
  Attempting uninstall: pandas
    Found existing installation: pandas 1.3.4
    Not uninstalling pandas at /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/MPI/gcc/11.2.0/openmpi/4.1.1/scipy-bundle/2021.10/lib/python3.9/site-packages, outside environment /projects/academic/ccradmintest/dsajdaktest/hail
    Can't uninstall 'pandas'. No files were found to uninstall.
    Running setup.py install for pyspark ... done
  Attempting uninstall: plotly
    Found existing installation: plotly 5.12.0
    Not uninstalling plotly at /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcccore/11.2.0/plotly.py/5.12.0/lib/python3.9/site-packages, outside environment /projects/academic/ccradmintest/dsajdaktest/hail
    Can't uninstall 'plotly'. No files were found to uninstall.

but those can be ignored. Once the installation completes, then you can test to see if the software works for you. If it needed the updated versions of the plotly, numpy and pandas that it tried to uninstall in those lines I showed above, then this virtual environment may not be the right option for you. I would then recommend you use Easybuild to create a python bundle, as described here.

[la-mendicino]

Thank you @dsajdak! I did create the virtual environment yesterday after I saw the easybuild recipe was not available. I did some initial testing on it and it worked properly, will do some more once the cooling outage is finished.

I also found this easybuild recipe from the Fred Hutch. Thanks again!

[dsajdak]

@la-mendicino oh, good find! I'm happy to try that one out. I'll have to modify some of the dependency version numbers but that may still work. I'll keep you posted.