search

maho3 / ltu-cmass

A modular simulator of CMASS-NGC galaxy clustering
6 stars 1 forks source link

Test installation and implementation of the current ltu-cmass pipeline #12

Closed maho3 closed 2 months ago

maho3 commented 3 months ago

The goal of this task is to have new users work through the installation instructions of ltu-cmass to see if they can get the full pipeline working on their machines.

Currently, the nbodykit installation has only been tested for Infinity and Rusty, though it would be useful if instructions can be written for other machines (NERSC, Beluga, etc.)

Definition of done

maho3 commented 3 months ago

Tagging volunteers @Xiaosheng-Zhao, @NiallJeffrey, @LaurencePeanuts

NiallJeffrey commented 3 months ago

Why does the documentation suggest git clone git@github.com:maho3/ltu-cmass.git rather than git clone https://github.com/maho3/ltu-cmass.git ? For me the first didn't work by default.

LaurencePeanuts commented 3 months ago

Yes, same here, I had to use git clone git@github.com/maho3/ltu-cmass.git

Xiaosheng-Zhao commented 3 months ago

For me, git clone https://github.com/maho3/ltu-cmass.git is also the only one that works by default.

LaurencePeanuts commented 3 months ago

We're having Cython issues for egg_info installation on our cluster... seems like the issue is with kdcount, we don't have the package and not sure what version of python is required for it.

NiallJeffrey commented 3 months ago

I can't get Nbodykit to install on NERSC: NERSC has very specific requirements to install mpi4py (https://docs.nersc.gov/development/languages/python/parallel-python/#mpi4py-in-your-custom-conda-environment)

module swap PrgEnv-${PE_ENV,,} PrgEnv-gnu
MPICC="cc -shared" pip install --force-reinstall --no-cache-dir --no-binary=mpi4py mpi4py

This MPICC seems to clash with pfft

File "/tmp/pip-install-59oq_9ow/pfft-python_fdc61aa0698b4dab9fb08501e884cb7d/setup.py", line 30, in build_pfft
          raise ValueError("could not build fftw; check MPICC?")
      ValueError: could not build fftw; check MPICC?

The nbodykit documentation has NERSC examples, but they are all outdated (pre-perlmutter). I'll ask around to see if anyone has solved this...

maho3 commented 2 months ago

It seems the core of these problems are in installing nbodykit, but once nbodykit is installed, the rest is fairly straightforward. @suicee and Sammy managed to get it installed on Infinity and Narval, and I've included their notes in the documentation.

With the completion of PR #25 and the opening of Issue #24, I'll close this issue.