weiji14 / deepbedmap

Going beyond BEDMAP2 using a super resolution deep neural network. Also a convenient flat file data repository for high resolution bed elevation datasets around Antarctica.
GNU Lesser General Public License v3.0
43 stars 26 forks source link

Upgrade from Python 3.6 to 3.7 #162

Closed weiji14 closed 5 years ago

weiji14 commented 5 years ago

Lots of new goodies in Python 3.7! Mainly nice to haves e.g. better parallelization with improved asyncio usability via asyncio.run(), easier debugging using breakpoint(), etc. Most of this DeepBedMap's dependencies have supported Python 3.7 for months already, all except onnx-chainer but that's changed with https://github.com/chainer/onnx-chainer/pull/192. Woohoo!

TODO:

weiji14 commented 5 years ago

Could have updated netcdf4 from 1.4.1 to 1.5.1.2, but that would require compiling the netcdf4-python package from source. No easy way to specify that in the Pipfile, but we could use the PIP_NO_BINARY environment variable (see this Stack Overflow post) and do it like so:

PIP_NO_BINARY=netCDF4 pipenv install netcdf4==1.5.1.2

but that won't be accurately reflected in the Pipfile.lock (?) that would undermine reproducibility as compiling the package at different times (with different compilers) may yield slightly different netcdf4 library products :frowning_face:

Wait, but sticking to netcdf==1.4.1 and using python 3.7 still means we are compiling it (that's why we see this and also why $HDF5_DIR was set) :man_facepalming: , but at least we don't need to modify our Dockerfile to set a new $PIP_NO_BINARY environment variable... Anyways, too out of scope for this Pull Request, so pushing that forward to the future.