pangeo-data / pangeo-example-notebooks

Pangeo Example Notebooks
106 stars 53 forks source link

Adds trimesh_hurricane example and hvplot graphics for Landsat and SSH examples #4

Closed rsignell-usgs closed 6 years ago

rsignell-usgs commented 6 years ago

Add Hurricane Ike trimesh example from pangeo.esipfed.org

martindurant commented 6 years ago

You may want to tag people you think would be best at reviewing this.

rsignell-usgs commented 6 years ago

@ocefpaf, can you please review, perhaps by logging into pangeo.esipfed.org and trying it out? You should find it in the examples folder.

ocefpaf commented 6 years ago

LGTM. I ran it without any problems.

I wonder if there is any xarray magic alternative to the near function you use there. It looks like .sel won't work b/c x and y are not indexes (dims), only node. (This is a place where where iris data model is better than xarray.)

BTW, I usually OK with jet for temperature data but you can probably use a better colormap there.

jacobtomlinson commented 6 years ago

Came here to also raise the use of jet. Ping @niallrobinson

rabernat commented 6 years ago

Does this work on pangeo.pydata.org?

rsignell-usgs commented 6 years ago

It does if you install geoviews:

2018-08-23_11-42-49

I installed geoviews from the pyviz dev channel because the geoviews from the defaults channel wanted to downgrade my netcdf.

jovyan@jupyter-rsignell-2dusgs:~$ conda install -c pyviz/label/dev geoviews
Solving environment: done

## Package Plan ##

  environment location: /opt/conda

  added / updated specs:
    - geoviews

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    geos-3.6.2                 |       heeff764_2         1.6 MB  defaults
    libspatialindex-1.8.5      |       h20b78c2_2         666 KB  defaults
    poppler-0.65.0             |       h581218d_1         1.6 MB  defaults
    click-plugins-1.0.3        |           py36_0          10 KB  defaults
    openssl-1.0.2p             |       h14c3975_0         3.5 MB  defaults
    geoviews-core-1.5.4a5      |             py_0         344 KB  pyviz/label/dev
    libkml-1.3.0               |       h590aaf7_4         633 KB  defaults
    cligj-0.4.0                |           py36_0          14 KB  defaults
    gdal-2.2.2                 |   py36hc209d97_1         767 KB  defaults
    pyproj-1.9.5.1             |   py36h7b21b82_1          64 KB  defaults
    libdap4-3.19.1             |       h6ec2957_0         1.5 MB  defaults
    lxml-4.2.4                 |   py36hf71bdeb_0         1.6 MB  defaults
    kealib-1.4.7               |       h77bc034_6         170 KB  defaults
    munch-2.3.2                |           py36_0          13 KB  defaults
    descartes-1.1.0            |           py36_0           9 KB  defaults
    giflib-5.1.4               |       h14c3975_1          78 KB  defaults
    pysal-1.14.4.post1         |           py36_1        14.9 MB  defaults
    psycopg2-2.7.5             |   py36hb7f436b_0         294 KB  defaults
    libpq-10.5                 |       h1ad7b7a_0         2.7 MB  defaults
    libxslt-1.1.32             |       h1312cb7_0         538 KB  defaults
    krb5-1.16.1                |       hc83ff2d_6         1.4 MB  defaults
    geoviews-1.5.4a5           |             py_0           3 KB  pyviz/label/dev
    proj4-5.0.1                |       h14c3975_0         7.0 MB  defaults
    shapely-1.6.4              |   py36h7ef4460_0         326 KB  defaults
    cartopy-0.16.0             |   py36hfa13621_0         1.7 MB  defaults
    owslib-0.16.0              |           py36_0         235 KB  defaults
    poppler-data-0.4.9         |                0         3.5 MB  defaults
    libgdal-2.2.4              |       h6f639c0_1        16.1 MB  defaults
    xerces-c-3.2.1             |       hac72e42_0         3.2 MB  defaults
    pyshp-1.2.12               |           py36_0          35 KB  defaults
    fiona-1.7.12               |   py36h3f37509_0         704 KB  defaults
    libboost-1.67.0            |       h46d08c1_4        20.9 MB  defaults
    freexl-1.0.5               |       h14c3975_0          44 KB  defaults
    openjpeg-2.3.0             |       h05c96fa_1         456 KB  defaults
    rtree-0.8.3                |           py36_0          46 KB  defaults
    pyepsg-0.3.2               |           py36_0          12 KB  defaults
    geopandas-0.3.0            |           py36_0         924 KB  defaults
    json-c-0.13.1              |       h1bed415_0          70 KB  defaults
    libspatialite-4.3.0a       |      he475c7f_19         3.1 MB  defaults
    ------------------------------------------------------------
                                           Total:        90.7 MB

The following NEW packages will be INSTALLED:

    cartopy:         0.16.0-py36hfa13621_0  defaults
    click-plugins:   1.0.3-py36_0           defaults
    cligj:           0.4.0-py36_0           defaults
    descartes:       1.1.0-py36_0           defaults
    fiona:           1.7.12-py36h3f37509_0  defaults
    freexl:          1.0.5-h14c3975_0       defaults
    gdal:            2.2.2-py36hc209d97_1   defaults
    geopandas:       0.3.0-py36_0           defaults
    geos:            3.6.2-heeff764_2       defaults
    geoviews:        1.5.4a5-py_0           pyviz/label/dev
    geoviews-core:   1.5.4a5-py_0           pyviz/label/dev
    giflib:          5.1.4-h14c3975_1       defaults
    json-c:          0.13.1-h1bed415_0      defaults
    kealib:          1.4.7-h77bc034_6       defaults
    krb5:            1.16.1-hc83ff2d_6      defaults
    libboost:        1.67.0-h46d08c1_4      defaults
    libdap4:         3.19.1-h6ec2957_0      defaults
    libgdal:         2.2.4-h6f639c0_1       defaults
    libkml:          1.3.0-h590aaf7_4       defaults
    libpq:           10.5-h1ad7b7a_0        defaults
    libspatialindex: 1.8.5-h20b78c2_2       defaults
    libspatialite:   4.3.0a-he475c7f_19     defaults
    libxslt:         1.1.32-h1312cb7_0      defaults
    lxml:            4.2.4-py36hf71bdeb_0   defaults
    munch:           2.3.2-py36_0           defaults
    openjpeg:        2.3.0-h05c96fa_1       defaults
    owslib:          0.16.0-py36_0          defaults
    poppler:         0.65.0-h581218d_1      defaults
    poppler-data:    0.4.9-0                defaults
    proj4:           5.0.1-h14c3975_0       defaults
    psycopg2:        2.7.5-py36hb7f436b_0   defaults
    pyepsg:          0.3.2-py36_0           defaults
    pyproj:          1.9.5.1-py36h7b21b82_1 defaults
    pysal:           1.14.4.post1-py36_1    defaults
    pyshp:           1.2.12-py36_0          defaults
    rtree:           0.8.3-py36_0           defaults
    shapely:         1.6.4-py36h7ef4460_0   defaults
    xerces-c:        3.2.1-hac72e42_0       defaults

The following packages will be UPDATED:

    openssl:         1.0.2o-h14c3975_1      defaults        --> 1.0.2p-h14c3975_0 defaults

Proceed ([y]/n)? y

Downloading and Extracting Packages
geos-3.6.2           |  1.6 MB | ##################################################################################### | 100%
libspatialindex-1.8. |  666 KB | ##################################################################################### | 100%
poppler-0.65.0       |  1.6 MB | ##################################################################################### | 100%
click-plugins-1.0.3  |   10 KB | ##################################################################################### | 100%
openssl-1.0.2p       |  3.5 MB | ##################################################################################### | 100%
geoviews-core-1.5.4a |  344 KB | ##################################################################################### | 100%
libkml-1.3.0         |  633 KB | ##################################################################################### | 100%
cligj-0.4.0          |   14 KB | ##################################################################################### | 100%
gdal-2.2.2           |  767 KB | ##################################################################################### | 100%
pyproj-1.9.5.1       |   64 KB | ##################################################################################### | 100%
libdap4-3.19.1       |  1.5 MB | ##################################################################################### | 100%
lxml-4.2.4           |  1.6 MB | ##################################################################################### | 100%
kealib-1.4.7         |  170 KB | ##################################################################################### | 100%
munch-2.3.2          |   13 KB | ##################################################################################### | 100%
descartes-1.1.0      |    9 KB | ##################################################################################### | 100%
giflib-5.1.4         |   78 KB | ##################################################################################### | 100%
pysal-1.14.4.post1   | 14.9 MB | ##################################################################################### | 100%
psycopg2-2.7.5       |  294 KB | ##################################################################################### | 100%
libpq-10.5           |  2.7 MB | ##################################################################################### | 100%
libxslt-1.1.32       |  538 KB | ##################################################################################### | 100%
krb5-1.16.1          |  1.4 MB | ##################################################################################### | 100%
geoviews-1.5.4a5     |    3 KB | ##################################################################################### | 100%
proj4-5.0.1          |  7.0 MB | ##################################################################################### | 100%
shapely-1.6.4        |  326 KB | ##################################################################################### | 100%
cartopy-0.16.0       |  1.7 MB | ##################################################################################### | 100%
owslib-0.16.0        |  235 KB | ##################################################################################### | 100%
poppler-data-0.4.9   |  3.5 MB | ##################################################################################### | 100%
libgdal-2.2.4        | 16.1 MB | ##################################################################################### | 100%
xerces-c-3.2.1       |  3.2 MB | ##################################################################################### | 100%
pyshp-1.2.12         |   35 KB | ##################################################################################### | 100%
fiona-1.7.12         |  704 KB | ##################################################################################### | 100%
libboost-1.67.0      | 20.9 MB | ##################################################################################### | 100%
freexl-1.0.5         |   44 KB | ##################################################################################### | 100%
openjpeg-2.3.0       |  456 KB | ##################################################################################### | 100%
rtree-0.8.3          |   46 KB | ##################################################################################### | 100%
pyepsg-0.3.2         |   12 KB | ##################################################################################### | 100%
geopandas-0.3.0      |  924 KB | ##################################################################################### | 100%
json-c-0.13.1        |   70 KB | ##################################################################################### | 100%
libspatialite-4.3.0a |  3.1 MB | ##################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
jacobtomlinson commented 6 years ago

@rsignell-usgs rainbow is also not acceptable (I thought it was just another name for jet). I recommend you explore colour maps without discontinuities such as the brewer maps.

Also see this article

rsignell-usgs commented 6 years ago

I understand that rainbow is not perceptually correct. But I also know that if I use viridis or magma I can't see the variability. The colorbar is right there if people want to see the actual value.

I'm going to continue to use the colormap I selected in demos I give. I guess pangeo can change it here if they want.

jacobtomlinson commented 6 years ago

I guess the point is are you actually seeing variability, or are you seeing an artifact of the colour map.

ocefpaf commented 6 years ago

BTW, I usually OK with jet for temperature data but you can probably use a better colormap there.

When I made this comment I meant it to be taken with a grain of salt. I understand all the issues with jet, rainbow, etc. I actually started a perceptually correct set of colormaps for a really old project a long time ago back in 2006, before viridis and cmocean, never got people to adopt it though b/c there was not a "community will" to change at the time. Ironically my "bad" colormaps are far more popular. (Those are from my MatLab days and I guess some are actually @rsignell-usgs colormaps.)

With that said, I would like to emphasize that I am OK with jet-like colormaps for oceanographic variables, specially for SST. There are very few perceptually "correct colomaps" that show the nice hot-to-cold feel. It is important to note that these variables are not brain scans, where artifacts may be a problem with your science.

BTW, I never saw any oceanographic science related issue with a jet-colored SST, we use the actual values for science, the image is only to illustrate the big picture.

(Just read a paper on meso-scale turbulence with black-n-white SST images and a single jet-colored version of the averaged images. I did not find any problem trusting the science just b/c the images where horrible. Again, the science was done with the numbers and not the images.)

TL;DR let's avoid the herd-phenomena of bashing jet and leave some room for freedom of choice in a notebook where the main goal is to demonstrate tri-mesh manipulation :wink:

I myself am committed to better images but I don't think the use of jet or rainbow is as terrible as people are making it to be.

rabernat commented 6 years ago

So the only thing holding this up from working is installing geoviews on pangeo.pydata.org.

Is there an issue for that?

rsignell-usgs commented 6 years ago

I've updated this PR and tested that everything runs on the pangeo binder: Binder

I stripped the output from my hurricane trimesh notebook, and added hvplot interactive graphics to the sea level rise notebook, which I think is pretty cool:

2018-10-18_13-32-54

2018-10-18_13-06-54

To have these run on pangeo.pydata.org, we would need to add at least hvplot and geoviews.

Note that we construct the environment.yml here a bit differently than on pangeo.pydata.org's Dockerfile. On the recommendation of @ocefpaf, we specify only the conda-forge channel at the top, and then specify the specific packages that unfortunately still need the custom intake or pyviz channels.