PeterDSteinberg
commented
7 years ago The experimental Elm-Earthio-NLDAS repo has some examples of Grib file usage in the loading of NLDAS forcing and VIC data.
Open PeterDSteinberg opened 7 years ago
PeterDSteinberg
commented
7 years ago The experimental Elm-Earthio-NLDAS repo has some examples of Grib file usage in the loading of NLDAS forcing and VIC data.
PeterDSteinberg
commented
7 years ago Note NCAR has a dev build of pynio for Python 3 (see comments on pynio issue issue #10 )
conda create -n (your_env_name) -c ncar -c conda-forge pynio=dev python=3
PeterDSteinberg
commented
7 years ago Hi @gbrener - sometime in next ~1 week could you create a pynio environment and try out your NLDAS visualization notebooks again (with the new pynio mentioned above and your notebook with widgets in the top level examples dir). This Grib file support and/or documentation of work arounds is needed for AGU prep (Elm-Earthio-NLDAS repo issues/PRs).
gbrener
commented
7 years ago Sure, no problem.
PeterDSteinberg
commented
7 years ago @gbrener [partial thread drift here] - I'm trying out the pynio dev build for 3.* as well as pydap (both are part of the NLDAS AGU stuff but I can stick with Py 2.7 if we have trouble).
Do you think:
pynio should be in Earthio's conda.recipe?
pydap should be in Earthio's conda.recipe?Are they stable enough? Somehow optional or specific to Python 2? I'll post my conda env creation and trial and error here later today or Monday.
PeterDSteinberg
commented
7 years ago Ok I had success setting up a conda environment for Py 3.6 with the dev build of pynio and pydap. Here's what I did (this is geared towards getting the AGU work in Elm-Earthio-NLDAS PR 4 working):
git clone https://github.com/bokeh/datashader/
cd datashader/examples && conda env create --file environment.yml --name elm-36 && cd ..
source activate elm-36
conda install -c elm/label/dev -c conda-forge earthio elm && conda install -c ncar -c conda-forge pynio=dev
# Make sure you're on Elm PR 221 - https://github.com/ContinuumIO/elm/pull/221
conda remove elm;pip uninstall -y elm;
conda install -c conda-forge pydap
cd elm && python setup.py develop && cd ../earthio && python setup.py develop && cd ../xarray_filters && python setup.py develop && cd ..
# Make sure you're on this branch - https://github.com/dask/dask-searchcv/pull/61
cd dask_searchcv && python setup.py develop && cd ..
cd Elm-Earthio-NLDAS && python nldas_soil_moisture_ml.pyAnd here's what that created:
conda listzaffine 2.1.0 py_1 conda-forge
appnope 0.1.0 py36_0 conda-forge
asn1crypto 0.22.0 py36_0 conda-forge
attrs 17.2.0 py36_0 conda-forge
backports 1.0 py36_1 conda-forge
backports.functools_lru_cache 1.4 py36_1 conda-forge
beautifulsoup4 4.6.0 py36_0 conda-forge
biggus 0.15.0 py36_0 conda-forge
blas 1.1 openblas conda-forge
bleach 2.0.0 py36_0 conda-forge
bokeh 0.12.9 py36_0 conda-forge
boost 1.64.0 py36_4 conda-forge
boost-cpp 1.64.0 1 conda-forge
boto3 1.4.7 py36_0 conda-forge
botocore 1.5.92 py36_0 conda-forge
bottleneck 1.2.1 py36_1 conda-forge
bzip2 1.0.6 1 conda-forge
ca-certificates 2017.7.27.1 0 conda-forge
cachey 0.1.1 <pip>
cairo 1.14.6 4 conda-forge
cartopy 0.15.1 py36_4 conda-forge
certifi 2017.7.27.1 py36_0 conda-forge
cf_units 1.1.3 py36_1 conda-forge
cffi 1.10.0 py36_0 conda-forge
chardet 3.0.4 py36_0 conda-forge
click 6.7 py36_0 conda-forge
click-plugins 1.0.3 py36_0 conda-forge
cligj 0.4.0 py36_0 conda-forge
cloudpickle 0.4.0 py36_0 conda-forge
colorcet 1.0.0 py36h86301b6_0 bokeh
cryptography 2.0.3 py36_0 conda-forge
curl 7.54.1 0 conda-forge
cycler 0.10.0 py36_0 conda-forge
cython 0.27.2 py36_0 conda-forge
dask 0.15.4 py_0 conda-forge
dask-core 0.15.4 py_0 conda-forge
datashader 0.6.2 py36_0 bokeh
datashape 0.5.4 py36_0 conda-forge
deap 1.0.2.post2 py36_0 conda-forge
decorator 4.1.2 py36_0 conda-forge
dill 0.2.6 py36_0 conda-forge
distributed 1.19.3 py36_0 conda-forge
docopt 0.6.2 py36_0 conda-forge
docutils 0.14 py36_0 conda-forge
earthio 0.0.1 py36_51 elm/label/dev
entrypoints 0.2.3 py36_1 conda-forge
expat 2.1.0 3 conda-forge
fastparquet 0.1.3 py36_0 conda-forge
flake8 3.5.0 py36_0 conda-forge
fontconfig 2.12.1 4 conda-forge
freetype 2.7 1 conda-forge
freexl 1.0.2 2 conda-forge
g2clib 1.6.0 3 conda-forge
geos 3.5.1 1 conda-forge
geoviews 1.3.2 py36_0 ioam
gettext 0.19.8.1 0 conda-forge
giflib 5.1.4 0 conda-forge
glib 2.51.4 0 conda-forge
graphviz 2.38.0 5 conda-forge
gsw 3.2.0 py36_0 conda-forge
gunicorn 19.7.1 py36_0 conda-forge
h5netcdf 0.5.0 py_0 conda-forge
h5py 2.7.1 py36_1 conda-forge
hdf4 4.2.12 0 conda-forge
hdf5 1.8.18 1 conda-forge
hdfeos2 2.19.1 7 conda-forge
hdfeos5 5.1.15 7 conda-forge
heapdict 1.0.0 py36_0 conda-forge
holoviews 1.8.4 py36h6f73470_0 ioam
html5lib 0.999999999 py36_0 conda-forge
hypothesis 3.23.0 py36_0 conda-forge
icu 58.1 1 conda-forge
idna 2.6 py36_1 conda-forge
imageio 2.2.0 py36_0 conda-forge
ipykernel 4.6.1 py36_0 conda-forge
ipython 6.2.1 py36_0 conda-forge
ipython_genutils 0.2.0 py36_0 conda-forge
ipywidgets 7.0.3 py_0 conda-forge
iris 1.13.0 py36_1 conda-forge
jasper 1.900.1 4 conda-forge
jedi 0.10.2 py36_0 conda-forge
jinja2 2.9.6 py36_0 conda-forge
jmespath 0.9.3 py36_0 conda-forge
jpeg 9b 1 conda-forge
json-c 0.12.1 0 conda-forge
jsonschema 2.6.0 py36_0 conda-forge
jupyter 1.0.0 py36_0 conda-forge
jupyter_client 5.1.0 py36_0 conda-forge
jupyter_console 5.2.0 py36_0 conda-forge
jupyter_core 4.3.0 py36_0 conda-forge
jupyter_dashboards 0.7.0 py36_0 conda-forge
kealib 1.4.7 3 conda-forge
krb5 1.14.2 0 conda-forge
libdap4 3.18.3 2 conda-forge
libffi 3.2.1 3 conda-forge
libgdal 2.1.4 3 conda-forge
libgfortran 3.0.0 0 conda-forge
libiconv 1.14 4 conda-forge
libkml 1.3.0 1 conda-forge
libmo_unpack 3.0 1 conda-forge
libnetcdf 4.4.1.1 7 conda-forge
libpng 1.6.28 1 conda-forge
libpq 9.6.3 0 conda-forge
libsodium 1.0.10 0 conda-forge
libspatialite 4.3.0a 15 conda-forge
libssh2 1.8.0 1 conda-forge
libtiff 4.0.7 0 conda-forge
libuuid 1.0.3 1 conda-forge
libxml2 2.9.5 0 conda-forge
libxslt 1.1.29 5 conda-forge
llvmlite 0.20.0 py36_0
locket 0.2.0 py36_1 conda-forge
lxml 4.1.0 py36_0 conda-forge
markupsafe 1.0 py36_0 conda-forge
marshmallow 2.14.0 <pip>
matplotlib 2.1.0 py36_0 conda-forge
mccabe 0.6.1 py36_0 conda-forge
mechanicalsoup 0.8.0 py_0 conda-forge
mistune 0.7.4 py36_0 conda-forge
mo_pack 0.2.0 py36_1 conda-forge
msgpack-python 0.4.8 py36_0 conda-forge
multipledispatch 0.4.9 py36_0 conda-forge
nbconvert 5.3.1 py_1 conda-forge
nbformat 4.4.0 py36_0 conda-forge
nc_time_axis 1.0.2 py36_0 conda-forge
ncurses 5.9 10 conda-forge
netcdf4 1.3.0 py36_0 conda-forge
networkx 2.0 py36_0 conda-forge
notebook 5.2.0 py36_1 conda-forge
numba 0.35.0 np113py36_0
numexpr 2.6.4 py36_0 conda-forge
numpy 1.13.3 py36_blas_openblas_200 [blas_openblas] conda-forge
olefile 0.44 py36_0 conda-forge
openblas 0.2.19 2 conda-forge
openjpeg 2.1.2 2 conda-forge
openssl 1.0.2l 0 conda-forge
owslib 0.15.0 py_0 conda-forge
pandas 0.20.3 py36_1 conda-forge
pandoc 1.19.2 0 conda-forge
pandocfilters 1.4.1 py36_0 conda-forge
param 1.5.1 py36_0 ioam
paramnb 2.0.2 py36_0 ioam
partd 0.3.8 py36_0 conda-forge
patsy 0.4.1 py36_0 conda-forge
pcre 8.39 0 conda-forge
pexpect 4.2.1 py36_0 conda-forge
pickleshare 0.7.4 py36_0 conda-forge
pillow 4.3.0 py36_0 conda-forge
pip 9.0.1 py36_0 conda-forge
pixman 0.34.0 0 conda-forge
poppler 0.52.0 2 conda-forge
poppler-data 0.4.7 0 conda-forge
proj4 4.9.3 4 conda-forge
prompt_toolkit 1.0.15 py36_0 conda-forge
psutil 5.4.0 py36_0 conda-forge
ptyprocess 0.5.2 py36_0 conda-forge
py 1.4.34 py36_0 conda-forge
pycodestyle 2.3.1 py36_0 conda-forge
pycparser 2.18 py36_0 conda-forge
pydap 3.2.1 py36_0 conda-forge
pyepsg 0.3.2 py36_0 conda-forge
pyflakes 1.6.0 py36_0 conda-forge
pygments 2.2.0 py36_0 conda-forge
pyke 1.1.1 py36_0 conda-forge
pympler 0.5 py36_0 conda-forge
pynio dev_20170921 05806a2_py36 ncar
pyopenssl 17.2.0 py36_0 conda-forge
pyparsing 2.2.0 py36_0 conda-forge
pyproj 1.9.5.1 py36_0 conda-forge
pyqt 5.6.0 py36_4 conda-forge
pyshp 1.2.12 py_0 conda-forge
pysocks 1.6.7 py36_0 conda-forge
pytables 3.4.2 py36_5 conda-forge
pytest 3.2.3 py36_0 conda-forge
pytest-benchmark 3.0.0 py36_1 conda-forge
python 3.6.3 0 conda-forge
python-dateutil 2.6.1 py36_0 conda-forge
python-graphviz 0.8 py36_0 conda-forge
python-snappy 0.5.1 py36_2 conda-forge
pytz 2017.2 py36_0 conda-forge
pywavelets 0.5.2 np113py36_0 conda-forge
pyyaml 3.12 py36_1 conda-forge
pyzmq 16.0.2 py36_2 conda-forge
qt 5.6.2 h9e3eb04_4 conda-forge
qtconsole 4.3.1 py36_0 conda-forge
rasterio 0.36.0 py36_0 conda-forge
readline 6.2 0 conda-forge
requests 2.18.4 py36_1 conda-forge
s3transfer 0.1.11 py36_0 conda-forge
scikit-image 0.13.0 py36_2 conda-forge
scikit-learn 0.19.1 py36_blas_openblas_200 [blas_openblas] conda-forge
scipy 0.19.1 py36_blas_openblas_202 [blas_openblas] conda-forge
setuptools 36.6.0 py36_1 conda-forge
shapely 1.6.1 py36_1 conda-forge
simplegeneric 0.8.1 py36_0 conda-forge
singledispatch 3.4.0.3 py36_0 conda-forge
sip 4.18 py36_1 conda-forge
six 1.11.0 py36_1 conda-forge
snappy 1.1.7 1 conda-forge
snuggs 1.4.1 py36_0 conda-forge
sortedcontainers 1.5.7 py36_0 conda-forge
sqlite 3.13.0 1 conda-forge
statsmodels 0.8.0 py36_0 conda-forge
streamz 0.1.0 py36_0 conda-forge
tblib 1.3.2 py36_0 conda-forge
terminado 0.6 py36_0 conda-forge
testpath 0.3.1 py36_0 conda-forge
thrift 0.10.0 py36_0 conda-forge
tk 8.5.19 2 conda-forge
toolz 0.8.2 py_2 conda-forge
tornado 4.5.2 py36_0 conda-forge
traitlets 4.3.2 py36_0 conda-forge
udunits2 2.2.25 1 conda-forge
urllib3 1.22 py36_0 conda-forge
wcwidth 0.1.7 py36_0 conda-forge
webargs 1.8.1 <pip>
webencodings 0.5 py36_0 conda-forge
webob 1.7.1 py36_0 conda-forge
wheel 0.30.0 py_1 conda-forge
widgetsnbextension 3.0.6 py36_0 conda-forge
xarray 0.9.6 py36_0 conda-forge
xerces-c 3.1.4 3 conda-forge
xz 5.2.3 0 conda-forge
yaml 0.1.6 0 conda-forge
zeromq 4.2.1 1 conda-forge
zict 0.1.3 py_0 conda-forge
zlib 1.2.8 3 conda-forge
zope 1.0 py36_0
zope.interface 4.4.2 py36_0pip freezeaffine==2.1.0
appnope==0.1.0
asn1crypto==0.22.0
attrs==17.2.0
backports.functools-lru-cache==1.4
beautifulsoup4==4.6.0
Biggus==0.15.0
bleach==2.0.0
bokeh==0.12.9
boto3==1.4.7
botocore==1.5.92
Bottleneck==1.2.1
cachey==0.1.1
Cartopy==0.15.1
certifi==2017.7.27.1
cf-units==1.1.3
cffi==1.10.0
chardet==3.0.4
click==6.7
click-plugins==1.0.3
cligj==0.4.0
cloudpickle==0.4.0
colorcet==1.0.0
cryptography==2.0.3
cycler==0.10.0
Cython==0.27.2
dask==0.15.4
-e git+https://github.com/dask/dask-searchcv@ce34117a23c4ecc2543b32683bd6787102960ced#egg=dask_searchcv
datashader==0.6.2
datashape==0.5.4
deap==1.0.2
decorator==4.1.2
dill==0.2.6
distributed==1.19.3
docopt==0.6.2
docutils==0.14
earthio==0.0.1+51.gfd83fdc
-e git+https://github.com/ContinuumIO/elm@b422e68f910c3ecc5db653c2d05a93b501064059#egg=elm
entrypoints==0.2.3
fastparquet==0.1.3
flake8==3.5.0
geoviews==1.3.2
graphviz==0.8
gsw==3.2.0
gunicorn==19.7.1
h5netcdf==0.5.0
h5py==2.7.1
heapdict==1.0.0
holoviews==1.8.4
html5lib==0.999999999
hypothesis==3.23.0
idna==2.6
imageio==2.2.0
ipykernel==4.6.1
ipython==6.2.1
ipython-genutils==0.2.0
ipywidgets==7.0.3
Iris==1.13.0
jedi==0.10.2
Jinja2==2.9.6
jmespath==0.9.3
jsonschema==2.6.0
jupyter-client==5.1.0
jupyter-console==5.2.0
jupyter-core==4.3.0
jupyter-dashboards==0.7.0
llvmlite==0.20.0
locket==0.2.0
lxml==4.1.0
MarkupSafe==1.0
marshmallow==2.14.0
matplotlib==2.1.0
mccabe==0.6.1
MechanicalSoup==0.8.0
mistune==0.7.4
mo-pack==0.2.0
msgpack-python==0.4.8
multipledispatch==0.4.9
nbconvert==5.3.1
nbformat==4.4.0
nc-time-axis==1.0.2
netCDF4==1.3.0
networkx==2.0
notebook==5.2.0
numba==0.35.0
numexpr==2.6.4
numpy==1.13.3
olefile==0.44
OWSLib==0.15.0
pandas==0.20.3
pandocfilters==1.4.1
param==1.5.1
paramnb==2.0.2
partd==0.3.8
patsy==0.4.1
pexpect==4.2.1
pickleshare==0.7.4
Pillow==4.3.0
prompt-toolkit==1.0.15
psutil==5.4.0
ptyprocess==0.5.2
py==1.4.34
pycodestyle==2.3.1
pycparser==2.18
Pydap==3.2.1
pyepsg==0.3.2
pyflakes==1.6.0
Pygments==2.2.0
pyke==1.1.1
Pympler==0.5
PyNIO===dev-20170921-05806a2
pyOpenSSL==17.2.0
pyparsing==2.2.0
pyproj==1.9.5.1
pyshp==1.2.12
PySocks==1.6.7
pytest==3.2.3
pytest-benchmark==3.0.0
python-dateutil==2.6.1
python-snappy==0.5.1
pytz==2017.2
PyWavelets==0.5.2
PyYAML==3.12
pyzmq==16.0.2
qtconsole==4.3.1
rasterio==0.36.0
requests==2.18.4
s3transfer==0.1.11
scikit-image==0.13.0
scikit-learn==0.19.1
scipy==0.19.1
Shapely==1.6.1
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.11.0
snuggs==1.4.1
sortedcontainers==1.5.7
statsmodels==0.8.0
streamz==0.1.0
tables==3.4.2
tblib==1.3.2
terminado==0.6
testpath==0.3.1
thrift==0.10.0
toolz==0.8.2
tornado==4.5.2
traitlets==4.3.2
urllib3==1.22
wcwidth==0.1.7
webargs==1.8.1
webencodings==0.5
WebOb==1.7.1
widgetsnbextension==3.0.6
xarray==0.9.6
-e git+https://github.com/ContinuumIO/xarray_filters/@bb2bbcef54f3b5f89f60fb7d161062c2e392bca1#egg=xarray_filters
zict==0.1.3
zope.interface==4.4.2
See also
earthioissue on Grib - we need to document how to use Grib files with Elm. When doing so, check on the Python 3 support bypygriband mention its status.