Make casa6 package

[gijzelaerr]

CASA moved away from the monolith design containing a python interpreter, and is now distributed as a binary wheel:

https://casa.nrao.edu/casadocs/casa-5.6.0/introduction/casa6-installation-and-usage

This is potentially good news. I couldn't find instructions on how to make my own binary wheel though. For now, I did some experiments using wheel2deb:

https://pypi.org/project/wheel2deb/

which does yield a Debian package, but I run into issues with incompatible numpy versions.

I'll use this issue to keep track of any progress.

[gijzelaerr]

I found the casa 6 build instructions.

https://casa.nrao.edu/casadocs-devel/stable/casa-development-team/development-resources/build-configuration/packaging-developer-builds

to build:

sudo apt install libeigen3-dev python3-venv
git clone --recursive https://open-bitbucket.nrao.edu/scm/casa/casa6
cd casa6
python3 -m venv venv
source venv/bin/activate
pip install numpy matplotlib scipy wheel
cd casatools   
scripts/gcw-pick     
autoconf             
./configure          
./setup.py build     

but the final step fails on my system (Ubuntu 20.04) with:

Casacore version 3.2.1
Traceback (most recent call last): 
  File "setup.py", line 2405, in <module>
    build_grpc( os.path.join('build', distutils_dir_name('lib'), module_name, '__casac__'),
  File "setup.py", line 703, in build_grpc
    create_ccache_bin( )
  File "setup.py", line 631, in create_ccache_bin
    write_wrapper( broken_autoconf_compiler_dir, props['build.compiler.ccache'], c )
  File "setup.py", line 614, in write_wrapper
    os.chmod(join(dir,os.path.basename(compiler)), 0o0755)
FileNotFoundError: [Errno 2] No such file or directory: '/home/gijs/Work/casa/casa6/casatools/build/compilers.linux-x86_64-3.8/gcc'

[gijzelaerr]

I reported the issue upstream also:

https://help.nrao.edu/index.php?/Tickets/Ticket/View/16365

but unfortunately, their issue tracker is non-public.

[twillis449]

I'm running Ubuntu 20.04 on a 'prototyping' machine to make sure there aren't any left-over python2 libraries that might get dragged into my system. Sounds like KERN is not quite ready for 20.04 or is it? Thanks Tony

[gijzelaerr]

Kern-7 will target 20.04

Op za 27 jun. 2020 02:36 schreef Tony Willis notifications@github.com:

I'm running Ubuntu 20.04 on a 'prototyping' machine to make sure there aren't any left-over python2 libraries that might get dragged into my system. Sounds like KERN is not quite ready for 20.04 or is it? Thanks Tony

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kernsuite/packaging/issues/227#issuecomment-650463483, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACPVJB4F5ZF4GQJZLEYV7LRYU5GLANCNFSM4N2DY6NA .

[r-xue]

Hi Gijs,

I created a Docker base image which packs casa6 under Ubuntu 20.04. The documentation (still a work-in-progress) and source files are available here: https://github.com/r-xue/casa6-docker

The original purpose of this repository was to help the distribution of another visibility simulation / modeling package I am developing. That modeling package depends on casatools (previously casacore) for MS table I/O, although the rest parts (e.g., FT, invert, etc.) are not, and any intermediate data are actually saved to HDF5. On the other hand, I also wanted a Docker image, so I can run expensive MCMC jobs on a HPC system with Singularity offered on my campus.

In any case, although my approach is a little bit hack (see some limited documentation there), I hope the source files from casa6-docker (likely the Dockerfile) might be helpful for the release of Kern-7.

[gijzelaerr]

hi @r-xue thanks for taking the effort to share this.

Unfortunately, this doesn't seem to actually build CASA, something we need to do to build a proper package. What I also tried was to convert the wheel into a debian package using https://pypi.org/project/wheel2deb/. This almost worked, but the numpy and scipy versions required by casa were not available in the Debian archive. What are your experiences here? Did you manage to get the numpy/scipy/matplotlib bundled with ubuntu 20.04 to work with CASA, or do you have to use the pip installed ones?

thanks again!

[r-xue]

No. To minimize the task, I don't build casa6 binary. My script basically uses casatools .whl released by NRAO (only for macOS/Linux under Py36 at this moment) and does some "hacking" jobs to get it installed under Py37/Py38. The script also works on macOS10.15 which is my default development OS.

Yes. I get everything (IPython, Jupyter, Astropy, etc. via pip, not from the Debian archive) packaged in 20.04 with the workflow illustrated in the Dockerfile. Some minimal comments are there. In 20.04, I had to manually install some .so files (e.g. libgfortran.so.3) since the binary casatools needs them. On the other hand, macOS is okay after the .whl hacking. I am still testing the setup though... But fo my modeling code, some basic functions from casatools are enough.

More background info is here. It's really not an elegant approach. I just want to get the whole thing working quickly on the cluster with Singularity.

[gijzelaerr]

I think we by now all agree that this packaging path is just too much effort unless the compatibility upstream changes at some point. For now I'm going to close this issue.