search

NorESMhub / noresm2cmor

A command line tool for cmorizing NorESM output
http://noresmhub.github.io/noresm2cmor/
5 stars 16 forks source link

zg500 (AERday): different members have identical values #78

Open lisesg opened 4 years ago

lisesg commented 4 years ago

Describe the bug I just got an email for a PAMIP collaborator analyzing the zg500 data that there are some strange features in the CMOR-ized zg500 field. The NorESM2 has noisy behaviour when sinuosity (waviness) is plotted as a function of time. One thing that has been identified is that several members have identical values. I have confirmed this, and checked that this is not the case with the raw model output.

To Reproduce To see that values are identical for different members in the CMOR-ized output, compare values for members 2 and 3 from CMOR-ized pdSST-pdSIC:

To see that values are non-identical for two members in the raw model output, compare values for members 2 and 3 from raw model output from pdSST-pdSIC:

YanchunHe commented 4 years ago

I will look at this right now.

lisesg commented 4 years ago

Another thing is that there seems to be differences between for instance CMOR-ized zg500 for pdSST-pdSIC member 2 and the raw Z500 for member 2. Should member 2 in the raw data correspond to member 2 in the CMOR-ized data? To reproduce (make sure to look as the same date):

YanchunHe commented 4 years ago

Do other monthly fields have such similar problems in addition to the daily zg500?

YanchunHe commented 4 years ago

A quick check that other monthly fields, e.g., ts in Amon are different for member 2 and 3.

lisesg commented 4 years ago

I have the same impression, that is, I have not been able to find the same problem for other fields yet.

YanchunHe commented 4 years ago

I think I found where the problem is. I updated the format of name list files slightly for your last several experiments, but not the first two experiments in the beginning. So that my script to submit the cmorization jobs does not set correct member tags for the v20191009 version (in which AERday is included).

So, for the six PAMIP experiments:

Should only the earliest first two cmorized experiments were affected:

I will redo the cmorization.

Could @monsieuralok @IngoBethke retracted fields from ESGF, or we just publish a newer version?

Effected experiments:

Effected fields:

Including all the 100 ensemble members

YanchunHe commented 4 years ago

Fixed in the v20191018 version. put in the queue to publish to ESGF. refer #28

YanchunHe commented 4 years ago

fix to the namelist files are committed to: 02e6773c11dcdb71f3001ec3c53d0d9863ae8101

lisesg commented 4 years ago

Great news! Thanks a lot! :)

lisesg commented 1 month ago

It seems like the corrected data was never published on ESGF? I just got an email from a user who rediscovered this bug, and when I do a search on ESGF, I can only find the old version (v20191009).

The updated version (v20191018) exists on NIRD.

lisesg commented 1 month ago

Another issue is that members 101-200 appear to be missing in the updated version: #182 and #183

YanchunHe commented 2 weeks ago

It seems like the corrected data was never published on ESGF? I just got an email from a user who rediscovered this bug, and when I do a search on ESGF, I can only find the old version (v20191009).

The updated version (v20191018) exists on NIRD.

Yes, this is strange but indeed it was not published.

I updated some metadata, so that it should be able publish them right now.

I will update their individual experiment (refer to #28).

YanchunHe commented 2 weeks ago

Another issue is that members 101-200 appear to be missing in the updated version: #182 and #183

I will check this too.

YanchunHe commented 2 weeks ago

Another issue is that members 101-200 appear to be missing in the updated version: #182 and #183 @lisesg

I don't see there are daily output in the extended experiments.

Could you please check that?

lisesg commented 2 weeks ago

Thanks for looking into this @YanchunHe! :)

The daily output is in the atmospheric h1 files. These seem to exist for all the experiments when I do a simple ls. For instance: ls /projects/NS2345K/noresm/cases/NFHISTnorpddmsbc_PAMIP_1p6_pdSST-futArcSIC_mem176-200_f19_mg17_20200313/atm/hist/*h1*

YanchunHe commented 2 weeks ago

Thanks for looking into this @YanchunHe! :)

The daily output is in the atmospheric h1 files. These seem to exist for all the experiments when I do a simple ls. For instance: ls /projects/NS2345K/noresm/cases/NFHISTnorpddmsbc_PAMIP_1p6_pdSST-futArcSIC_mem176-200_f19_mg17_20200313/atm/hist/*h1*

OK, thanks! I looked into wrong place. Now I find it.

I will update the #182 and #183 issues.