search

ECCO-GROUP / ECCOv4-py

A Python library with routines that support the loading, analysis, and plotting fields of the ECCO Version 4 Ocean and Sea-Ice State Estimate. The ecco_v4_py library builds on several valuable tools such as xmitgcm, gcm, xarray, and dask.
MIT License
30 stars 30 forks source link

basemap vs. cartopy #5

Closed ifenty closed 5 years ago

ifenty commented 5 years ago

This package shoud migrate to Cartopy and leave Basemap for geographically-aware plotting because Basemap is going away.

timothyas commented 5 years ago

I definitely agree. Additionally I'd like to throw out another option, which it might be possible to pursue in tandem. I have only played with this a little bit, but it might be worth considering geoviews. It's a package built on the library holoviews, which seems to make the data easier to visualize and play with, while making the meta data accessible in the plots (so they claim...).

This seems like an extra good idea because holo/geo views works with datashader, which aims to make visualization of large datasets scalable. @rabernat pointed me in this direction, with the idea that it would be nice to have one data visualization pipeline for llc90 -> llc4320 grids.

I started to play with this for MITgcm output, specifically for ECCO, and you can check it out here. This uses xmitgcm which I think is a win. I really only started this, you can see I didn't even stitch the llc faces together yet, because then I got "distracted" by my thesis proposal... At the bottom I tried integrating it with datashader, but to no avail yet. But, this could be a start.

Anyway, I'm curious what you think! It also seems to me like the group @xecco should be merged into the @ECCO-GROUP (or vice versa etc ...) so as to not duplicate any efforts here.

ifenty commented 5 years ago

datashader looks interesting but probably overkill for the time being for llc90/llc270 simulations Geoviews looks to be immediately useful being that it is built on cartopy. I looked at your notebook but didn't find any images.

As for your point about xecco, do you know how to do a merge into ecco-group?

timothyas commented 5 years ago

I agree with what you said, my hope is that using geoviews will set us up for integrating with datashader easier down the line. But I'll be honest it's just a hypothesis, you'll see I didn't manage to do it yet although I have not spent a ton of time with it yet.

I made just a static html page which you can download here. Again, it's really nothing at all. It was more of a to-do list and just shows some of the nice features of geoviews (interactivity, slider bars etc), which I think you'd get a better idea by checking out their webpage.

As for xecco, I'm not sure if there is a formal way to do that. In my humble opinion, we at xecco should probably move to the ECCO-GROUP repo because I don't think we have done much over there (at least on the plotting side) and there are only a couple of us. @slmcbane and I had just started working on plotting then got taken away by grad school commitments, and @maikejulie was working on vector calculus routines. My suggestion would be to have these efforts could be absorbed into existing or new ECCO-GROUP projects.

PS any reason for everything by ecco being in ALL CAPS? :P

slmcbane commented 5 years ago

I don't think there's really any work from xecco that would be a great loss - we can just delete the group and join this one.

maikejulie commented 5 years ago

Hi all,

I agree we should move away from basemap.

I just got approached by Martin who has a Gradstudent that's done some nice work using cartoply, great timing!

More fancy and fast plotting of big datasets would be wonderful, and I encourage work on that direction.

Re xecco, I think Ian's work, along with Raffaele and what I did should all be merged.

Cheers!

elenamarie commented 5 years ago

Hi, I am Elena the Gradstudent of Martin;) I am also not an experienced cartopy user - I just played a bit around with xmitgcm and plotting llc90 data with cartopy. Here is an example.

I tried plotting the data directly (without interpolation), which works fine with pcolormesh - even for the Arctic - but unfortunately it fails when plotting with contourf. So I think interpolating the data first to a regular grid as you did in your library is maybe more stable. And I think it should be not too difficult to migrate your plotting functions from Basemap to cartopy...

But you should consider that cartopy is still under development and there are still many bugs...

ifenty commented 5 years ago

Hi Elana, When you say plotting "directly" without interpolation you still had to add the extra rows and columns, correct? I haven't tested but I'm not sure that is faster than interpolating to lat/lon. I suppose it probably depends on the resolution of the lat/lon. For the llc90 it's usually enough to go to 1/3 degree and capture everything.

The ecco-v4-py routines are specifically meant to read the netcdf files that we provide as part of the 'official release'. In the next release, I'm advocating that the 12 lat-lon tiles are all 'rotated' properly and that the fields on the 'u','v', and 'g' points are output as tiles of size 91x90, 90x91, and 91x91 so that one can load a 90x90 tracer tile and its corresponding 'u' field and have 'u' values on both the west and east faces for each grid point.

I did try cartopy first and gave up last year because of bugs - whether they were mine or theirs I don't know. I'm going to give it another try.

elenamarie commented 5 years ago

Yes, plotting 'directly' included the addition of rows/columns to every side of the faces in order to close the gaps;-)

I tried to plot my data loaded with xmitgcm into an xarray with your plot function and it works more ore less. I added it in the same jupyter notebook below: example notebook I had to copy the function into my notebook because I am using python 3 and I needed to change the print statements. Unfortunately, the data are only plotted completely when I set user_lon_0 = 180 and plot_type = 'contourf' also produces a pcolormesh plot. I could not find out yet if it has something to do with my data or with python 3 vs. 2.7 or if I changed something in the function which is causing these bugs...

Plotting without interpolation seems to be a bit faster. - at least for the llc90 resolution. But I guess interpolating the data first is more stable, so this is probably the better approach.

elenamarie commented 5 years ago

I changed the plot_tiles_proj function in a way to be able to use cartopy and apart from some warnings it is working quite well (with the new cartopy version 0.17.0). I only tested it for my model output (loaded with xmitgcm) yet, but maybe some of you could test it with the ECCO netcdf output?!? You can find my notebook with some example plots here.

I only included five of the cartopy projection types but if necessary there could be introduced others (here the list with possible cartopy projection types). And I removed the plotting in two parts 'A' and 'B' because it always resulted in only partly filled plots and it worked well when plotting it all together.

The only thing which is not working yet, is plotting with a polar stereographic projection with a central_longitude other than 0 (look at the end of the notebook).

ifenty commented 5 years ago

Elena,

I’ll take a look! Thanks!!

Ian

From: elenamarie notifications@github.com Sent: Monday, November 19, 2018 2:21 PM To: ECCO-GROUP/ECCOv4-py ECCOv4-py@noreply.github.com Cc: Ian Fenty ifenty@gmail.com; Author author@noreply.github.com Subject: Re: [ECCO-GROUP/ECCOv4-py] basemap vs. cartopy (#5)

I changed the plot_tiles_proj function in a way to be able to use cartopy and apart from some warnings it is working quite well (with the new cartopy version 0.17.0). I only tested it for my model output (loaded with xmitgcm) yet, but maybe some of you could test it with the ECCO netcdf output?!? You can find my notebook with some example plots here https://github.com/elenamarie/llc90_example_data/blob/master/plot_llc90_with_eccopy_and_cartopy.ipynb .

I only included five of the cartopy projection types but if necessary there could be introduced others (here https://scitools.org.uk/cartopy/docs/latest/crs/projections.html the list with possible cartopy projection types). And I removed the plotting in two parts 'A' and 'B' because it always resulted in only partly filled plots and it worked well when plotting it all together.

The only thing which is not working yet, is plotting with a polar stereographic projection with a central_longitude other than 0 (look at the end of the notebook).

— You are receiving this because you authored the thread. Reply to this email directly, https://github.com/ECCO-GROUP/ECCOv4-py/issues/5#issuecomment-440062939 view it on GitHub, or https://github.com/notifications/unsubscribe-auth/ALOAVcd6odEFalPGDZdc-9f_C6JroI0yks5uwy7WgaJpZM4YPm0x mute the thread. https://github.com/notifications/beacon/ALOAVcdPT_IfW32AyNQxcL5L6fPDJGrvks5uwy7WgaJpZM4YPm0x.gif

ifenty commented 5 years ago

projected plotting routines all now use cartopy thanks to the good suggestions of @elenamarie .

there are now three routines: plot_proj_to_latlon_grid plot_global plot_pstereo

plot_proj_to_latlon_grid takes a field at arbitrary lat/lon points (e.g., ECCO output) and projects the field onto a lat/lon grid. the projection is either to a polar stereographic map or one of several 'global' maps.

plot_global takes a gridded field at gridded x/y points (can be lat/lon or some other projection like epsg:3413) and plots the field onto some arbitrary cartopy projection axis defined by the variable 'ax'

plot_pstereo is just like plot_global except that it has some extra features for plotting on polar stereographic axes.

the routines aren't well documented yet at least plot_proj_to_latlon_grid should be pretty self-explanatory.

thanks for the suggestions!