Missing data for some domains after clipping

eucp-project / data-catalogue

EUCP data portal

https://eucp-project.github.io/data-catalogue/

Other

1 stars 0 forks source link

Missing data for some domains after clipping #38

Open geek-yang opened 2 years ago

geek-yang commented 2 years ago

In the "Analyse" tab of cpm page, for some regions in the cpm maps, some data points are missing after clipping. This is due to the in-complete data coverage in some regions. The data is preprocessed by Tom (Crocker). We need the original (raw) data to perform the clipping and plotting the maps.

Peter9192 commented 2 years ago

@geek-yang on your local build you showed this morning, the maps at least showed up (although largely filled with nan/masked data). On the main page they don't even show. Can you make them show up, and upload an example image in this issue to clarify the problem?

Peter9192 commented 2 years ago

Ah,, I see https://github.com/eucp-project/data-catalogue/pull/39 now. Good.

Peter9192 commented 2 years ago

So for example:

where you can see that the majority of the domain is missing (set to zero to be precise).

Similarly:

Peter9192 commented 2 years ago

The data is preprocessed by Tom (Crocker). We need the original (raw) data to perform the clipping and plotting the maps.

Well, let's not jump to conclusions.. @thomascrocker can you remind us how you determined the bounds of the domains? We obtained these images by clipping the data you provided to the domain extents from the deliverable report.

geek-yang commented 2 years ago

@geek-yang on your local build you showed this morning, the maps at least showed up (although largely filled with nan/masked data). On the main page they don't even show. Can you make them show up, and upload an example image in this issue to clarify the problem?

Hi Peter, sorry that I hadn't pushed those maps to the remote when talking to you this morning. But then I realized that you've merged that branch already (I was in the Analytics SIG). Therefore I created a separate PR for it. And thanks a lot for placing these maps here.

thomascrocker commented 2 years ago

I think what has happened here is the model files being plotted are for CPMs from different domains to the one that the plot is for.

It looks like a plot for the CEE-3 domain. For this domain, I only have two EUCP CPMs,

cordex-cpm_ICTP-RegCM4-7 CEE-3_anom_JJA.nc
cordex-cpm_SMHI-HCLIM38-AROME CEE-3_anom_JJA.nc

For these CPMs the domain covered is contained in the filename.

My scripts use functionality in our ASCEND shapefile library to calculate the percentage area of a shapefile that is covered by a given cube, and from that whether to include the provided cube of data in onward processing. See for example: https://github.com/eucp-project/Lines-of-evidence-catalog/blob/0ac29c1f74f32c38f6baa20b79d1342c1076840f/diagnostics/summary_boxplots.py#L214