Closed aimalz closed 9 months ago
Some notes I took for running rail pipeline on NERSC
Install rail:
module load python git clone https://github.com/LSSTDESC/rail.git cd rail mamba env create -f environment.yml -n [env] # or mamba env create, which is much faster conda activate [env] pip install . rail clone-source --package-file rail_packages.yml rail install --package-file rail_packages.yml --from-source
add the rail kernel to jupyter:
conda activate [env] # . ./cenv/bin/activate in case of virtualenv ([env])$ conda install ipykernel ([env])$ ipython kernel install --user --name=
([env])$ conda deactivate
If you run into issue with ceci or pipeline.run() in jupyterlab, it is a path issue with the jupyter.
you need to ssh into ssh username@perlmutter.nersc.gov , and run
module load python conda activate [env] Open your your_rail_pipeline.yml, add “name: local” under the site section. Then run ceci your_rail_pipeline.yml
Additional changes we may want to make:
More abstract:
I've created a PR that addresses all but two of the requests in the first comment. If I could get some help from someone about what to include for the two NERSC-related requests, I would really appreciate it. I'm not familiar with what they are referring to.
16 is about user-facing instructions. I'm using this issue to centralize developer-facing onboarding documentation.
Suggestions from the retreat (list will be edited continuously):
rail --help
,rail-nb --help [path]
(suggest running on some notebooks to confirm it works)pip install
topip install -e .
ceci
path issuerail-nb
command (insert this link)