Python Environmental file [merge]

espg commented 2 months ago

This PR adds an environment.yml file for ParallelGamit. This file can be used by both pip for automating the install (i.e., future hosting of the ParallelGamit library on the pypi repository), and interactively by the developers and users to create reproducible production/development environments.

Use of this file is generally achieved by mamba and/or conda. For unity, mamba is available as a module which also includes a conda install.

To setup a fresh environment for the first time

mamba env create -f environment.yml
conda activate pg

To update an existing environment after modifying `environment.yml`

# make changes to environment.yml
conda deactivate # drop out of activated python env
mamba env update -n pg --file environment.yml
conda activate pg

To do

Besides the first couple of items below, many of these will (likely) be separate pull requests in the future, but adding them here now for awareness:

[x] Test environment for compatibility with current code base @eckendrick
[ ] Prune unneeded dependencies. The dependency list was generated from the current working python environment on unity using 'pip freeze', which lists all installed python packages in site-packages. However, some of the packages listed are dependencies for mainline requirements (i.e., pandas is probably pulling in h5py), and this environment.yml will be shorter/clearer if we can remove the dependencies of dependencies (which are managed by the referring package itself at install time).
[ ] Unpin (unneeded) version numbers. Right now, most everything has either an explicit version pinned, or a minimum version number specified. We would ideally only enforce this when actually needed so that the mamba/conda/pip installer are less constrained. (Note, we can still produce a conda-lock file for explicit reproducibility of production environments if needed, which would be separate from this environment.yml install file.)
[x] Update installation documentation; separate PR
[ ] Add ParallelGamit to pypi. This would enable users to run pip install ParallelGamit to install, rather than doing either a git clone or manual install from a downloaded archive. Also would enable control and separation between development and production versions; separate PR

eckendrick commented 2 months ago

I tested the new environment.yml with 3 different python scripts (StationInfoEdit, PlotETM, and ScanArchive -ppp) on u111 and all work normally. I just tested with pyParallelGamit.py and I get this error message ImportError: cannot import name '_fit_context' from 'sklearn.base' (/home/kendrick.42/.conda/envs/pg/lib/python3.10/site-packages/sklearn/base.py) conda list shows my scikit-learn is version 1.1.3

espg commented 2 months ago

@demiangomez this can probably be merged as is; items 4 and 5 are definitely separate PR's... items 2 and 3 could be added as part of those two PR's too...

demiangomez commented 2 months ago

@espg: can you please modify the readme also to add the instructions about installing and activating the environment?

mamba env create -f environment.yml conda activate pgamit

etc

I'll then merge this PR to main. Thanks

espg commented 2 months ago

@demiangomez this should be ready to merge now.

I updated INSTALL.md with instructions for installing the prerequisites-- I'll add more to that file after I setup and test the version releases in a separate PR.

Note that I removed the instructions for the proj dependency, since we aren't including that python package for the transforms... I also don't see any C code that would need the library path. (I also think that fiona / shapely / cartopy pull that base library and manage it internally already)

demiangomez / Parallel.GAMIT