NOAA-CSL / MELODIES-MONET

MELODIES MONET - diagnostic tool for evaluating models against a variety of observations including surface, aircraft, and satellite data all within a common framework
https://melodies-monet.readthedocs.io
Apache License 2.0
19 stars 21 forks source link

`domain_type == 'all'` behavior #140

Open zmoon opened 1 year ago

zmoon commented 1 year ago

There just might be some user confusion if the obs and model sets include points outside the spatial plotting domain which are then included in the other plots and statistics (which does happen with the current all behavior).

Originally posted by @mebruckner in https://github.com/NOAA-CSL/MELODIES-MONET/issues/107#issuecomment-1302641766

Currently with domain_type == 'all', all points get used in the spatial averages and stats. But in the spatial plots[^a], this setting causes the domain to be limited to CONUS, which is potentially misleading.

Furthermore, there isn't currently an easy way to get all points in the spatial plots to show (see https://github.com/NOAA-CSL/MELODIES-MONET/issues/107#issuecomment-1302630410).


I suppose we need a different way to specify CONUS (or e.g. the new, larger NA domain, or custom rectangular domain) for the spatial plots, and this domain restriction should also apply to the spatial averages and stats.

And with domain_type == 'all', the spatial plot behavior should be to set x/y lims to the max data extent.

[^a]: domain_type == 'all' where is commonly used with domain_name == 'CONUS'

zmoon commented 1 year ago

@mebruckner you said you already worked around this with your stuff, do you have any comments/suggestions?

mebruckner commented 1 year ago

I worked around the spatial plot issue by adding the domain_name tag to the plot extent determination, which then forced the code into the else portion which uses the max data extents.

https://github.com/NOAA-CSL/MELODIES-MONET/blob/19e523dff7b5fcad00d1586e4607aa912613fb5d/melodies_monet/plots/surfplots.py#L213-L228

mebruckner commented 1 year ago

I haven't done much with actually restricting the domain, other than adding custom region classifiers to surface observations that I can then filter on.

rschwant commented 1 year ago

@mebruckner I like your fix here.

for the domain_type == 'all'. This was originally meant to include all model data. No matter what the domain. I just added the extent here to make the plots look consistent for our regional analysis.

So I say we just update line 316, 664, and 1007 in the driver just to be: if domain_type == 'all' and domain_name == 'CONUS' like Maggie you did. Then maybe we should describe this in the ReadTheDocs too.

This is described here already briefly: https://melodies-monet.readthedocs.io/en/latest/appendix/yaml.html domain_name: List of domain names to be plotted. If domain_type = all, all data will be used and the domain_name is used only in the plot title. If domain_type is not equal to all, MELODIES MONET will query all of the data where domain_type is equal to domain_name.

Maybe we should add to this description something like: For “spatial_bias”, “spatial_overlay”, and “spatial_bias_exceedance” plots, if domain_type == 'all' and domain_name == 'CONUS' the following extent will be used for visual effect only: extent = [-130.0,-60.0,25.0,50.0]. This extent will not impact any calculations like spatial averages or statistics.

What do you all think about this?

zmoon commented 1 year ago

For “spatial_bias”, “spatial_overlay”, and “spatial_bias_exceedance” plots, if domain_type == 'all' and domain_name == 'CONUS' the following extent will be used for visual effect only: extent = [-130.0,-60.0,25.0,50.0]. This extent will not impact any calculations like spatial averages or statistics.

@rschwant I think this would resolve the issue, as long as it is made clear, maybe in an "important" or "note" box. We could also add other large rectangles like the new North America domain ('NA'?)? And maybe in the time series examples, e.g., show or stress that 'CONUS' doesn't need to be the domain_name for domain_type='all'.