Windows environment and testing pipeline

thodson-usgs commented 1 year ago

@kjdoore

@samstarzy is testing out the library and running into issues with the Windows environment. If you have some time to dig into this, I'd be very thankful.

I see this as a two part issue: 1) We need to update the requirements.txt such that we can get a working windows environment. This might be a little awkward, as windows and linux may require different configurations, and I don't know how to handle that. 2) Because we lack testing, we were't aware of the Windows issue. Once you have a working environment, add Windows to the testing pipeline, along with a couple basic tests the exercise most of the codebase: say, something that that fits a trivial example with advi and nuts and plots the results.

His environment is pasted below.

name: ratingCurve2
channels:
  - conda-forge
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - https://repo.anaconda.com/pkgs/msys2
  - defaults
dependencies:
  - arviz=0.16.1=pyhd8ed1ab_0
  - blas=1.0=mkl
  - bottleneck=1.3.5=py311h5bb9823_0
  - brotli=1.0.9=h0e60522_4
  - bzip2=1.0.8=he774522_0
  - ca-certificates=2023.05.30=haa95532_0
  - cachetools=5.3.1=pyhd8ed1ab_0
  - cloudpickle=2.2.1=pyhd8ed1ab_0
  - cons=0.4.6=pyhd8ed1ab_0
  - contourpy=1.0.5=py311h59b6b97_0
  - cycler=0.11.0=pyhd8ed1ab_0
  - etuples=0.3.9=pyhd8ed1ab_0
  - fastprogress=1.0.3=pyhd8ed1ab_0
  - filelock=3.12.2=pyhd8ed1ab_0
  - fonttools=4.25.0=pyhd3eb1b0_0
  - freetype=2.10.4=h546665d_1
  - h5netcdf=1.2.0=pyhd8ed1ab_0
  - h5py=3.7.0=py311h259cc0e_0
  - hdf5=1.10.6=nompi_he0bbb20_101
  - icc_rt=2022.1.0=h6049295_2
  - intel-openmp=2023.2.0=h57928b3_49496
  - jpeg=9e=h8ffe710_2
  - kiwisolver=1.4.4=py311hd77b12b_0
  - lerc=3.0=hd77b12b_0
  - libdeflate=1.17=h2bbff1b_0
  - libffi=3.4.4=hd77b12b_0
  - libpng=1.6.39=h8cc25b3_0
  - libtiff=4.5.0=h8a3f274_0
  - libwebp=1.2.4=h2bbff1b_0
  - libwebp-base=1.2.4=h2bbff1b_1
  - logical-unification=0.4.6=pyhd8ed1ab_0
  - lz4-c=1.9.3=h8ffe710_1
  - m2w64-binutils=2.25.1=5
  - m2w64-bzip2=1.0.6=6
  - m2w64-crt-git=5.0.0.4636.2595836=2
  - m2w64-gcc=5.3.0=6
  - m2w64-gcc-ada=5.3.0=6
  - m2w64-gcc-fortran=5.3.0=6
  - m2w64-gcc-libgfortran=5.3.0=6
  - m2w64-gcc-libs=5.3.0=7
  - m2w64-gcc-libs-core=5.3.0=7
  - m2w64-gcc-objc=5.3.0=6
  - m2w64-gmp=6.1.0=2
  - m2w64-headers-git=5.0.0.4636.c0ad18a=2
  - m2w64-isl=0.16.1=2
  - m2w64-libiconv=1.14=6
  - m2w64-libmangle-git=5.0.0.4509.2e5a9a2=2
  - m2w64-libwinpthread-git=5.0.0.4634.697f757=2
  - m2w64-make=4.1.2351.a80a8b8=2
  - m2w64-mpc=1.0.3=3
  - m2w64-mpfr=3.1.4=4
  - m2w64-pkg-config=0.29.1=2
  - m2w64-toolchain=5.3.0=7
  - m2w64-tools-git=5.0.0.4592.90b8472=2
  - m2w64-windows-default-manifest=6.4=3
  - m2w64-winpthreads-git=5.0.0.4634.697f757=2
  - m2w64-zlib=1.2.8=10
  - matplotlib-base=3.7.1=py311hf62ec03_1
  - minikanren=1.0.3=pyhd8ed1ab_0
  - mkl=2023.2.0=h6a75c08_49573
  - mkl-service=2.4.0=py311h2bbff1b_1
  - mkl_fft=1.3.6=py311hf62ec03_1
  - mkl_random=1.2.2=py311hf62ec03_1
  - msys2-conda-epoch=20160418=1
  - multipledispatch=0.6.0=py_0
  - munkres=1.1.4=pyh9f0ad1d_0
  - numexpr=2.8.4=py311h1fcbade_1
  - numpy=1.25.2=py311hdab7c0b_0
  - numpy-base=1.25.2=py311hd01c5d8_0
  - openssl=3.0.10=h2bbff1b_2
  - packaging=23.1=pyhd8ed1ab_0
  - pandas=1.5.3=py311heda8569_0
  - patsy=0.5.3=pyhd8ed1ab_0
  - pillow=9.4.0=py311hd77b12b_0
  - pip=23.2.1=py311haa95532_0
  - pymc=5.6.1=py311h5cc824b_0
  - pyparsing=3.1.1=pyhd8ed1ab_0
  - pytensor=2.12.3=py311hf62ec03_0
  - python=3.11.4=he1021f5_0
  - python-dateutil=2.8.2=pyhd8ed1ab_0
  - pytz=2023.3=pyhd8ed1ab_0
  - ratingcurve=0.3.5=pyhd8ed1ab_0
  - scipy=1.11.1=py311hc1ccb85_0
  - setuptools=68.0.0=py311haa95532_0
  - six=1.16.0=pyh6c4a22f_0
  - sqlite=3.41.2=h2bbff1b_0
  - tbb=2021.8.0=h59b6b97_0
  - tk=8.6.12=h2bbff1b_0
  - toolz=0.12.0=pyhd8ed1ab_0
  - typing-extensions=4.7.1=hd8ed1ab_0
  - typing_extensions=4.7.1=pyha770c72_0
  - tzdata=2023c=h04d1e81_0
  - vc=14.2=h21ff451_1
  - vs2015_runtime=14.27.29016=h5e58377_2
  - wheel=0.38.4=py311haa95532_0
  - xarray=2023.2.0=pyhd8ed1ab_0
  - xarray-einstats=0.6.0=pyhd8ed1ab_0
  - xz=5.4.2=h8cc25b3_0
  - zlib=1.2.13=h8cc25b3_0
  - zstd=1.4.9=h19a0ad4_0
prefix: C:\Users\Sam\anaconda3\envs\ratingCurve2

kjdoore commented 1 year ago

To Do's:

[x] Update requirements for Windows
[x] Add tests for basic usage: power law and spline fitting with ADVI and NUTS, table creation, loading datasets
[ ] Add fitting test with method='nuts' and core=1 if using Windows. Known PyMC issue when fitting with multiple cores on Windows.

thodson-usgs commented 1 year ago

Let's modify number 3. Something like:

[x] Add an environmental/global variable that sets a default number of cores. Upon import, check the os and set cores=1 for windows. Issue a warning message that windows is single threaded.

This way you won't need a separate unit test for windows.

thodson-usgs / ratingcurve

Windows environment and testing pipeline #31