Quick start installation instructions

OliviaLynn commented 1 year ago

On a call with @aimalz and @drewoldag, we observed that it would be nice to have a "quick start" section to the installation instructions for new users that want to hit the ground running.

Would also be a good exercise to make sure we can even condense a minimal set of installation instructions (or if installation process is too complicated for this).

jbkalmbach commented 1 year ago

Looking to try using RAIL for the first time in a while. Would it be useful for me to take this on as a new user and record what the minimum setup I need to get running would be?

OliviaLynn commented 1 year ago

Absolutely, that would be fantastic.

aimalz commented 1 year ago

I just wanted to leave a comment here summarizing a discussion that's been happening outside of GitHub, the consensus of which is that we actually want more than one quick start guide to which users will be directed from an initial starting point outlining the main ways we expect people to use RAIL, specifically providing different instructions for those who anticipate jumping straight to scalable pipelines versus those who will start by running exploratory tests in notebooks.

jbkalmbach commented 1 year ago

I can take it from an exploratory notebook perspective since that's my interest. Should we break this issue into different ones @aimalz?

eacharles commented 1 year ago

Have a look at my post in #desc-pz-rail about basic idea on how to get started. On May 22, 2023, at 12:41 PM, Bryce Kalmbach @.***> wrote: I can take it from an exploratory notebook perspective since that's my interest. Should we break this issue into different ones @aimalz?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

eacharles commented 1 year ago

Also, I put a zipped tar file on s3df with test versions of the various rail packages as laid out above.~echarles/xfer/rail_stuff.tgzHave a look. For the new packages I used the Lincc package template, but removed some stuff (docs & pylintrc files & a few src files that didn’t match up with typical rail usage) & put a few things in pyproject.toml On May 22, 2023, at 1:00 PM, Eric Charles @.> wrote:Have a look at my post in #desc-pz-rail about basic idea on how to get started. On May 22, 2023, at 12:41 PM, Bryce Kalmbach @.> wrote: I can take it from an exploratory notebook perspective since that's my interest. Should we break this issue into different ones @aimalz?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

aimalz commented 1 year ago

While we're at this, we should follow these instructions to ensure pz-rail and pz-rail-hub have non-blank project descriptions on PyPI. EDIT: And the same goes for updating that of qp-prob.

OliviaLynn commented 1 year ago

@eacharles, @sschmidt23 - Alex and I were just outlining some quickstart instructions for command line users. How do we envision users ultimately using rail_pipelines?

eg, I'm trying to run the goldenspike pipeline in rail_pipelines, so I:

$ python -m site
copy the (path-to-site-packages) that gets printed
$ ceci (path-to-site-packages)/rail/pipelines/examples/goldenspike/goldenspike.py

Is there a more streamlined route for the user to take? Is there (or will there be) some kind of command-line-accessible notion of rail's path?

eacharles commented 1 year ago

So, that I was imanging was that rail_pipelines would be a repository for two types of things.

python pipeline files, that allow you do load a pipeline into a python session or a notebook easily, run a few tests, and export the configuration to a yaml file
python yaml files for some standard pipelines, that you can use to run a notebook with ceci

For the first I think that there is an example notebook in rail_pipelines Basically you can do

<usual conda setup>
pip install pz-rail-pipelines
<Open a notebook, import a pipeline, play around with it, and save the config to pipe.yml>
ceci pipe.yml --input <some_big_file> --outdir <output>

for the second, I think you could do something like this:

git clone https://github.com/LSSTDESC/rail_pipelines.git
cd rail_pipelines
<usual conda setup>
pip install -e .
ceci src/rail/configs/goldenspike.yml --input <some_file> --outdir <some_dir>

Of course you could try and use python -m site to run with pip install pz-rail-pipelines but I would recommend doing as source install so that you can edit the pipeline config file as needed.

We need to decide now we want store the yml files for the pipelines. I.e., do they go somewhere in the src directory tree and get installed with the package.

OliviaLynn commented 1 year ago

Documenting some trouble (plus solutions) I ran into when installing RAIL locally on a summer research student's laptop (an M1 Mac using Miniforge) -

The steps that ultimately worked:

conda create -y -n rail python=3.10 h5py healpy scikit-learn somoclu
conda activate rail
pip install pz-rail-hub
conda install numpy
conda install scipy
pip install astropy==5.3

Reasons:

conda install h5py -> an issue with tables
conda install healpy -> issue with healpy from pip
pip install cmake -> arose from pip having trouble installing xgboost, but only needed to be run the one time
conda install numpy scipy -> assuming pz-rail-hub messes with the dependencies grabbed from prev conda scikit-learn. I don't fully understand but this was necessary when double checking these install instructions on her machine
upgrade astropy from 5.1 to 5.3 -> came from an issue running goldenspike: concatenate() got an unexpected keyword arg ‘dtype’

eacharles commented 1 year ago

Does this work:

mamba create -y -n rail python=3.10 h5py healpy scikit-learn somoclu numpy scipy astropy==5.3 conda activate rail git clone cd rail_hub pip install -e .

On Jun 13, 2023, at 11:11 AM, Olivia R. Lynn @.***> wrote:

Documenting some trouble (plus solutions) I ran into when installing RAIL locally on a summer research student's laptop (an M1 Mac using Miniforge) -

The steps that ultimately worked:

conda create -y -n rail python=3.10 h5py healpy scikit-learn somoclu conda activate rail pip install pz-rail-hub conda install numpy conda install scipy pip install astropy==5.3 Reasons:

conda install h5py -> an issue with tables conda install healpy -> issue with healpy from pip pip install cmake -> arose from pip having trouble installing xgboost, but only needed to be run the one time conda install numpy scipy -> assuming pz-rail-hub messes with the dependencies grabbed from prev conda scikit-learn. I don't fully understand but this was necessary when double checking these install instructions on her machine upgrade astropy from 5.1 to 5.3 -> came from an issue running goldenspike: concatenate() got an unexpected keyword arg ‘dtype’ — Reply to this email directly, view it on GitHub https://github.com/LSSTDESC/rail_hub/issues/18, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRIGIW3DK4IMSIZBNOWHCTXLCUOBANCNFSM6AAAAAAWZGZB7U. You are receiving this because you were mentioned.

eacharles commented 1 year ago

I've added some instructions:

https://lsstdescrail.readthedocs.io/en/latest/source/installation.html

I'm going to close this for now, as we have a number of open tickets about improving the documentation.

LSSTDESC / rail

Quick start installation instructions #18