MSWMS: Display coarse data with a lot of NaNs correctly

[soerenjoh]

I was trying to add MLS satellite data to my MSWMS Server and encountered the following problem:

• MLS satellite data is provided as level-2 data on orbits, which I gridded on daily maps on a regular lat/lon/pressure grid. Naturally, a lot of grid points are not filled with data (since interpolation and creation of "real" level-3 data is quite challenging and may impose artifacts). As a result, the netCDF variable contains a lot of NaNs.
• The grid sampling I chose was rather coarse, approximately in the range of the best horizontal resolution of the satellite (approx. 1.5°)

As a result, MSWMS shows this kind of data only very sparsely, I think due to the fact that it tries to interpolate the data on a finer grid, the NaNs are interpolated with the available data, which resulted again in a NaN. So only a few data points are shown in the horizontal map (at locations where orbits crossed and more points are available).

As a workaround for this issue, I used a finer grid and repeated all of the satellite measurements to the four neighboring lat/lon points. This helps to show the satellite data on kind-of orbits without too many plotting artifacts.

However, I think this issue should be avoided at all in case of coarse and patchy data, doesn't it? Maybe satellite retrospective data is not the primary data used to set up MSWMS, but I find it very handy to have all relevant data at hand for a campaign...

[ReimarBauer]

Is there missing_value or _FillValue attribute defined for the NaNs?

[soerenjoh]

Thanks for the hint! Using missing_value helped in the global horizontal view, but not after zooming in (????), adding _FillValue made everything worse again.

This is how the variable looks like in panoply:

And this is what MSWMS does with it:

[ReimarBauer]

Can you share a setup / data for debugging this whith us? @joernu76 can you have a look?

[soerenjoh]

I'll send a file to you and @joernu76. It's not big, but I'm not sure about copyright issues, since the data is not from me. Here are the entries of the mswms_settings.py file, which is used to create the plots:

mpl_hsec_styles.make_generic_class("HS_MyStyle_pl_hno3", "mole_fraction_of_nitric_acid_in_air", "pl", [], []) register_horizontal_layers = None if mpl_hsec_styles is not None: register_horizontal_layers = [ (mpl_hsec_styles.HS_MyStyle_pl_hno3, ["mls37"]), ]

Anything else you need?

[joernu76]

Mmmh. I am confused as MSWMS does not interpolate the data at all (for horizontal cross sections). Will have a look.

[joernu76]

Ah, I see the issue. The generic plots are doing a contourf plot. Strange that you get any data at all. You need a plotting class that uses pcolormesh or similar for plotting. See this example where I created something by copy-pasting and changing the contourf to pcolormesh. mswms_settings.py.txt I had to change the ending so github would let me attach the file.

[joernu76]

If this works, I will create a feature to allow the choice of either using contourf or pcolormesh for the generic plotting class in the develop branch.

[soerenjoh]

Thanks a lot @joernu76 ! Unfortunately, it's not working with version 7.0.8, which I have installed on my server. Generics is not available there... I'll try to find in the code how it looks like for this version. If possible, I would prefer not to change the installed version on the server, because it's running and working and I'm not sure why ;-)

[joernu76]

No probs. I can also provide a version for 7.0.8. But it is mostly copy-paste, renaming the class (which isn't actually necessary), exchanging the contourf by pcolormesh and putting an "extend" into the colorbar call, if I remember correctly.

[soerenjoh]

Thanks again. Here is the example of your file adjusted so it is working for version 7.0.8, just in case anyone else finds this issue: mswms_settings.py.txt Hopefully no one will need to use it, since you already are working on a fix :-)