Update repo scaffolding and conda test env

xCDAT / xcdat-validation

The xCDAT validation repository for exploring, prototyping, and testing feature ideas.

https://xcdat.readthedocs.io/en/latest/

3 stars 3 forks source link

Update repo scaffolding and conda test env #24

Closed tomvothecoder closed 3 years ago

tomvothecoder commented 3 years ago

Closes #19

Summary of Changes

Update README.md with "Getting Started" section
Replace conda env dev.yml with conda-env/test.yml
- Lists xcdat=0.1.0 now
- Lists cdutil and cdms2 as dependencies, not the entire CDAT library (high chance of package conflicts)
Update scaffolding of repo
- scripts/ for storing utility scripts (e.g., the PCMDI xml reader that Steve wrote)
- tutorials/ for any tutorials on how to use xarray and XCDAT -- may be moved to the official XCDAT docs in the future
- validation/ for storing CDAT vs. xarray/XCDAT comparison notebooks
Update existing notebooks
- Renamed files and stored in new directory scaffolding
- Remove test env setup sections inside notebooks since we use conda-env/test.yml now
Add VSCode project and workspace settings

tomvothecoder commented 3 years ago

Hey @pochedls and @lee1043, here's a PR for setting up the infrastructure of this repo. Let me know if I'm missing anything, such as CDAT dependencies required in conda-env/test.yml.

lee1043 commented 3 years ago

@tomvothecoder Some notebooks use pandas (e.g., this), should pandas to be included as one of dependencies?

tomvothecoder commented 3 years ago

@lee1043 Thanks Jiwoo, I have added them to test.yml.

numpy and pandas are implicitly installed through xarray and xcdat, but it is better to list them explicitly since they are major packages.

pochedls commented 3 years ago

Hey @pochedls and @lee1043, here's a PR for setting up the infrastructure of this repo. Let me know if I'm missing anything, such as CDAT dependencies required in conda-env/test.yml.

@tomvothecoder - I'm not sure if versioning matters, but I usually install CDAT with the following:

conda create -y -n cdat -c conda-forge -c cdat/label/v8.2.1 cdat "libnetcdf=*=mpi_openmpi_*" "mesalib=18.3.1" "python=3.7"

I think that this commit will work (I don't care about mesalib or mpi), but I am wondering if it uses Python 3.7 for a specific reason.

tomvothecoder commented 3 years ago

@pochedls I tried installing the entire cdat library with the dependency versions listed in test.yml (including Python 3.9), but ran into package conflict issues. As a workaround, only a subset of the CDAT libraries are listed (sourced from conda-forge).

Maybe Python 3.7 is listed in your command for compatibility reasons.