search

cedadev / cis

Home of the Community Intercomparison Suite.
www.cistools.net
GNU Lesser General Public License v3.0
46 stars 18 forks source link

Data collocated even when no data for variable is not available #21

Open tommibergman opened 5 years ago

tommibergman commented 5 years ago

With AERONET data there some stations that do not have any data for a given variable and year (there are other variables but not this one). Now I have an automatic script which collocates model data to AERONET. With earlier versions (at least 1.4) collocating to empty data would result in empty collocated data as well. But with v1.7.1 I am getting values for collocated data even when AERONET has no data. Of course I can do a work around, but I feel this is a reduction in usability.

adamcpovey commented 5 years ago

Could you provide a minimal example of the collocation you're running? I haven't encountered this problem but have a suspicion what it might be.

duncanwp commented 5 years ago

This can be controlled using the 'missing_data_for_missing_sample' keyword argument, are you using the Python or command line interface?

tommibergman commented 5 years ago

This can be controlled using the 'missing_data_for_missing_sample' keyword argument, are you using the Python or command line interface?

I have some older scripts which still use command line interface. Does this mean that I would get more control if I would use the Python?

duncanwp commented 5 years ago

Yes, I tend to only use the Python interface now, it's not as well documented yet but it gives you much more flexibility. See this example notebook to get you going :-)

tommibergman commented 5 years ago

Could you provide a minimal example of the collocation you're running? I haven't encountered this problem but have a suspicion what it might be.

I made a small test set with one month in my google drive https://drive.google.com/open?id=1sQftoLxgBbRkrpn0-x6DbaIGlOfyFccV .

running test from there will produce a subset of banizombou station and the colocate my data on it. And the collocated data has data while the subset does not.

tommibergman commented 5 years ago

I was also wondering why do they get different missing value? AERONET has -999 and model data NaN.

tommibergman commented 5 years ago

I was also wondering why do they get different missing value? AERONET has -999 and model data NaN.

Sorry, my mistake. This is just a difference between an old and a new version.