recommit all v2 changes with original untouched v1 metadata

[TonyB9000]

To revert all v1 metadata files to the v1 license, and not lose any of the corrected v2 metadata files, I sheltered all of the v2 files, deleted the local repo, git-cloned the master, added back the v2 files, and then committed.

[chengzhuzhang]

Hey @golaz , Tony and I have generated the metadata definition files for v2 LR simulations. Please feel free to review the E3SM-2-0 folder. To facilitate the review, Tony has created a text files which only included attribute that diff between different files. Please look here: https://github.com/E3SM-Project/CMIP6-Metadata/blob/3ddd060146166314f677ea40a10706585f55daaf/E3SM-2-0/check/proof Sorry to scramming your return-to-work task lists...

[TonyB9000]

When I "git-cloned the master, added back the v2 files, and then committed", I'm certain I did git add *.json and no other "adds", and yet it seemed to have included the contents of the "check" directory I was using. I "git rm'd" that.

[TonyB9000]

All Done. Will Push.

[TonyB9000]

Do we wait for Chris?

[chengzhuzhang]

@TonyB9000 could you please add back your "checker", so that the PR is easier to review by @golaz

[TonyB9000]

I think the "checker" was moved to /p/user_pub/e3sm/staging/resource/check_metadata/

Should I move it into the CMIP6-Metadata repo? It can be run where it is - but I don't know if you have access.

It runs against the "installed" metadata, not the Repo.

(and it only checks E3SM-2-0 material)

[chengzhuzhang]

@TonyB9000 Tony, I would suggest you to run the checker in this latest branch for E3SM-2-0 material, and post the results in the discussion. Alternatively, I think it is also okay to push the results in this branch.

[TonyB9000]

OK. But it occurs to me I should make a much nicer one (spreadsheet) with more fields of interest. This one merely shows "experiment_id", "history", "realization_index" and "branch_time_in_parent". I have a separate one the extracts all of the "license" info.

(base) -bash-4.2$ more proof ../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "experiment_id": "1pctCO2", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "experiment_id": "abrupt-4xCO2", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "experiment_id": "abrupt-4xCO2", ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "experiment_id": "amip", ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "experiment_id": "amip", ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "experiment_id": "amip", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "experiment_id": "hist-aer", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "experiment_id": "hist-aer", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "experiment_id": "hist-aer", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "experiment_id": "hist-aer", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "experiment_id": "hist-aer", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "experiment_id": "hist-GHG", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "experiment_id": "hist-GHG", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "experiment_id": "hist-GHG", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "experiment_id": "hist-GHG", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "experiment_id": "hist-GHG", ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "experiment_id": "historical", ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "experiment_id": "historical", ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "experiment_id": "historical", ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "experiment_id": "historical", ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "experiment_id": "historical", ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "experiment_id": "piClim-control", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "experiment_id": "piClim-histaer", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "experiment_id": "piClim-histaer", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "experiment_id": "piClim-histaer", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "experiment_id": "piClim-histall", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "experiment_id": "piClim-histall", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "experiment_id": "piClim-histall", ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "experiment_id": "piControl",

../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "history": "v2.LR.1pctCO2_0101", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "history": "v2.LR.abrupt-4xCO2_0101", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "history": "v2.LR.abrupt-4xCO2_0301", ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "history": "v2.LR.amip_0101", ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "history": "v2.LR.amip_0201", ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "history": "v2.LR.amip_0301", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "history": "v2.LR.hist-aer_0101", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "history": "v2.LR.hist-aer_0151", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "history": "v2.LR.hist-aer_0201", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "history": "v2.LR.hist-aer_0251", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "history": "v2.LR.hist-aer_0301", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "history": "v2.LR.hist-GHG_0101", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "history": "v2.LR.hist-GHG_0151", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "history": "v2.LR.hist-GHG_0201", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "history": "v2.LR.hist-GHG_0251", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "history": "v2.LR.hist-GHG_0301", ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "history": "v2.LR.historical_0101", ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "history": "v2.LR.historical_0151", ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "history": "v2.LR.historical_0201", ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "history": "v2.LR.historical_0251", ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "history": "v2.LR.historical_0301", ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "history": "v2.LR.piClim-control", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "history": "v2.LR.piClim-histaer_0021", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "history": "v2.LR.piClim-histaer_0031", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "history": "v2.LR.piClim-histaer_0041", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "history": "v2.LR.piClim-histall_0021", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "history": "v2.LR.piClim-histall_0031", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "history": "v2.LR.piClim-histall_0041", ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "history": "v2.LR.piControl",

../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "realization_index": "3", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "realization_index": "3", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "realization_index": "4", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "realization_index": "5", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "realization_index": "3", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "realization_index": "4", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "realization_index": "5", ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "realization_index": "3", ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "realization_index": "4", ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "realization_index": "5", ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "realization_index": "3", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "realization_index": "1", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "realization_index": "2", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "realization_index": "3", ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "realization_index": "1",

../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "branch_time_in_parent": 36500.0, ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "branch_time_in_parent": 36500.0, ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "branch_time_in_parent": 109500.0, ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "branch_time_in_parent": 36500.0, ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "branch_time_in_parent": 73000.0, ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "branch_time_in_parent": 109500.0, ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "branch_time_in_parent": 36500.0, ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "branch_time_in_parent": 54750.0, ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "branch_time_in_parent": 73000.0, ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "branch_time_in_parent": 91250.0, ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "branch_time_in_parent": 109500.0, ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "branch_time_in_parent": 36500.0, ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "branch_time_in_parent": 54750.0, ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "branch_time_in_parent": 73000.0, ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "branch_time_in_parent": 91250.0, ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "branch_time_in_parent": 109500.0, ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "branch_time_in_parent": 36500.0, ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "branch_time_in_parent": 54750.0, ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "branch_time_in_parent": 73000.0, ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "branch_time_in_parent": 91250.0, ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "branch_time_in_parent": 109500.0, ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "branch_time_in_parent": 0.0, ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "branch_time_in_parent": 7300.0, ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "branch_time_in_parent": 10950.0, ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "branch_time_in_parent": 14600.0, ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "branch_time_in_parent": 7300.0, ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "branch_time_in_parent": 10950.0, ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "branch_time_in_parent": 14600.0, ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "branch_time_in_parent": 0.0,

[TonyB9000]

I think we probably want "source_type", "parent_experiment_id", and "activity".

../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "source_type": "AOGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "source_type": "AGCM AER", ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "source_type": "AOGCM AER",

../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "activity_id": "DAMIP", ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "activity_id": "CMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "activity_id": "RFMIP AerChemMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "activity_id": "RFMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "activity_id": "RFMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "activity_id": "RFMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "activity_id": "RFMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "activity_id": "RFMIP", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "activity_id": "RFMIP", ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "activity_id": "CMIP",

../CMIP6-Metadata/E3SM-2-0/1pctCO2_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/abrupt-4xCO2_r2i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/amip_r1i1p1f1.json: "parent_experiment_id": "no parent", ../CMIP6-Metadata/E3SM-2-0/amip_r2i1p1f1.json: "parent_experiment_id": "no parent", ../CMIP6-Metadata/E3SM-2-0/amip_r3i1p1f1.json: "parent_experiment_id": "no parent", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r2i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r3i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r4i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-aer_r5i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r2i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r3i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r4i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/hist-GHG_r5i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/historical_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/historical_r2i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/historical_r3i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/historical_r4i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/historical_r5i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-control_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r2i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-histaer_r3i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r1i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r2i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piClim-histall_r3i1p1f1.json: "parent_experiment_id": "piControl", ../CMIP6-Metadata/E3SM-2-0/piControl_r1i1p1f1.json: "parent_experiment_id": "piControl-spinup",

[chengzhuzhang]

Yes, to extract the additional parameters:"source_type", "parent_experiment_id", and "activity". and to have a spreadsheet/csv file would be really useful. I think this utility can be useful for the large ensemble as well, but only "realization_index" and "branch_time_in_parent" would change in that case.

[TonyB9000]

Lets see if I can paste a table here:

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

experiment_id | parent_experiment_id | source_type | activity_id | history | realization_index | branch_time_in_parent -- | -- | -- | -- | -- | -- | -- 1pctCO2 | piControl | AOGCM AER | CMIP | v2.LR.1pctCO2_0101 | 1 | 36500 abrupt-4xCO2 | piControl | AOGCM AER | CMIP | v2.LR.abrupt-4xCO2_0101 | 1 | 36500 abrupt-4xCO2 | piControl | AOGCM AER | CMIP | v2.LR.abrupt-4xCO2_0301 | 2 | 109500 amip | no parent | AGCM AER | CMIP | v2.LR.amip_0101 | 1 | 36500 amip | no parent | AGCM AER | CMIP | v2.LR.amip_0201 | 2 | 73000 amip | no parent | AGCM AER | CMIP | v2.LR.amip_0301 | 3 | 109500 hist-aer | piControl | AOGCM AER | DAMIP | v2.LR.hist-aer_0101 | 1 | 36500 hist-aer | piControl | AOGCM AER | DAMIP | v2.LR.hist-aer_0151 | 2 | 54750 hist-aer | piControl | AOGCM AER | DAMIP | v2.LR.hist-aer_0201 | 3 | 73000 hist-aer | piControl | AOGCM AER | DAMIP | v2.LR.hist-aer_0251 | 4 | 91250 hist-aer | piControl | AOGCM AER | DAMIP | v2.LR.hist-aer_0301 | 5 | 109500 hist-GHG | piControl | AOGCM AER | DAMIP | v2.LR.hist-GHG_0101 | 1 | 36500 hist-GHG | piControl | AOGCM AER | DAMIP | v2.LR.hist-GHG_0151 | 2 | 54750 hist-GHG | piControl | AOGCM AER | DAMIP | v2.LR.hist-GHG_0201 | 3 | 73000 hist-GHG | piControl | AOGCM AER | DAMIP | v2.LR.hist-GHG_0251 | 4 | 91250 hist-GHG | piControl | AOGCM AER | DAMIP | v2.LR.hist-GHG_0301 | 5 | 109500 historical | piControl | AOGCM AER | CMIP | v2.LR.historical_0101 | 1 | 36500 historical | piControl | AOGCM AER | CMIP | v2.LR.historical_0151 | 2 | 54750 historical | piControl | AOGCM AER | CMIP | v2.LR.historical_0201 | 3 | 73000 historical | piControl | AOGCM AER | CMIP | v2.LR.historical_0251 | 4 | 91250 historical | piControl | AOGCM AER | CMIP | v2.LR.historical_0301 | 5 | 109500 piClim-control | piControl | AGCM AER | RFMIP AerChemMIP | v2.LR.piClim-control | 1 | 0 piClim-histaer | piControl | AGCM AER | RFMIP | v2.LR.piClim-histaer_0021 | 1 | 7300 piClim-histaer | piControl | AGCM AER | RFMIP | v2.LR.piClim-histaer_0031 | 2 | 10950 piClim-histaer | piControl | AGCM AER | RFMIP | v2.LR.piClim-histaer_0041 | 3 | 14600 piClim-histall | piControl | AGCM AER | RFMIP | v2.LR.piClim-histall_0021 | 1 | 7300 piClim-histall | piControl | AGCM AER | RFMIP | v2.LR.piClim-histall_0031 | 2 | 10950 piClim-histall | piControl | AGCM AER | RFMIP | v2.LR.piClim-histall_0041 | 3 | 14600 piControl | piControl-spinup | AOGCM AER | CMIP | v2.LR.piControl | 1 | 0

[TonyB9000]

Yes! I CAN!

[TonyB9000]

That was generated by /p/user_pub/e3sm/staging/resource/check_metadata/metadata_values_report.sh (CSV output, need to manually create the excel afterwards).

[chengzhuzhang]

Very awesome, @TonyB9000 @golaz hey, Chris, would you please double check the parameters that differ cross the v2 simulations. Tony listed all the parameters that varies. https://github.com/E3SM-Project/CMIP6-Metadata/pull/9#issuecomment-1246086256

Based on Tony's tests, these medata files work with cmorizing process.

Hopefully they are more straightforward to review. Please also check one of any .json files for common parameters. Note that we will add additional v2 LR reference paper once its DOI is available.

[chengzhuzhang]

@TonyB9000 The v2 overview paper lead by @golaz is now accepted and gets a DOI. Would you please update this PR by replacing:

"references": "Golaz, J.-C., P. M. Caldwell, L. P. Van Roekel and co-authors, 2019: The DOE E3SM coupled model version 1: Overview and evaluation at standard resolution. JAMES, doi: 10.1029/2018MS001603; http://e3sm.org'"

With "references": "Golaz, J.-C., L. P. Van Roekel, X. Zheng and co-authors, 2022: The DOE E3SM Model Version 2: Overview of the physical model and initial model evaluation. JAMES, doi: 10.1029/2022MS003156; http://e3sm.org'"

For all the simulations under v2?

[TonyB9000]

Sure. I'll find Charlie's "ncatted" (nc attribute editor) and find a way to run in against all the v2 metadata files. Then I'll push and PR.

(Sorry - I forgot these are just json files, not "in nc" metadata. The "ncatted" will need to be applied to ALL of the CMIP6 v2 we have already generated, prior to publication.)

[TonyB9000]

@chengzhuzhang The unmatched single-quote at the end of the reference (end of the URL) should be removed, I assume. That is what I've been doing...

[TonyB9000]

Done. Does anyone need to review this, or should I do the merge?

[chengzhuzhang]

No hurry to merge. This needs to be tested too.

[TonyB9000]

I created a separate branch (fix_pcode_in_mpas_py) to fix the mpaso/mpasocean disparity. I accidentally pushed it in the "preps_for_v2" branch (too many branches!) and need to un-push/un_PR. Not sure how.

[golaz]

Hi @TonyB9000, @chengzhuzhang

The table above looks good. A few more tweaks:

• The amip simulations were started from the corresponding historical simulations at year 1870 (for atmosphere, land and river components). We could indicate this by adding a parent_experiment_id and branch_time_in_parent.
• As we discussed with Jill, there are five additional simulations we should submit:
  • v2.LR.hist-all-xGHG-xaer_0101
  • v2.LR.hist-all-xGHG-xaer_0151
  • v2.LR.hist-all-xGHG-xaer_0201
  • v2.LR.hist-all-xGHG-xaer_0251
  • v2.LR.hist-all-xGHG-xaer_0301 They correspond approximately to DAMP hist-nat, but with slight deviation in the protocol, so we need to add a comment in the metadata:

"E3SMv2 hist-nat simulations have been configured to include all forcings except GHG and aer. As such, they differ slightly from the DAMIP specifications by including land-use changes and stratospheric ozone. This choice was made to include all forcings in the decomposition into GHG, aer, nat. Refer to the E3SMv2 overview manuscript for details."

[chengzhuzhang]

Hi @golaz Thank you for reviewing! Regarding to the amip parent_experiment_id, I looked it up before, in the CMIP vocabulary, the parent_experiment_id for amip should be no parent. https://wcrp-cmip.github.io/CMIP6_CVs/docs/CMIP6_experiment_id.html I'm tagging @durack1 to check if setting parent_experiment_id and branch_time_in_parent is possible for amip.

I will go ahead and add DAMP hist-nat, thank you for including a comment for the metadata!

[TonyB9000]

We have generated some 2000 v2 CMIP6 datasets over the past few months. We held off publication awaiting the updated references material. Once we had the updates, we used "ncatted" (nc attribute editor) to modify the metadata in the (9000+) files, and we published these to ESGF. Subsequently, we have generated the Lmon (and published), and are in the process of generating Omon and SImon (a slow process).

If there are metadata changes that MUST be included for particular datasets, we can hold off on publication of these. Otherwise we will need to retract and re-publish. We have already published all (non-Omon, non-SImon) material for amip. We spent considerable effort calculating "branch-time-in-parent" for all of these v2 experiments, so I hope they are correct. They were merged under a previous branch. The values obtained:

1pctCO2_r1i1p1f1.json:  "branch_time_in_parent": 36500.0,
abrupt-4xCO2_r1i1p1f1.json:  "branch_time_in_parent": 36500.0,
abrupt-4xCO2_r2i1p1f1.json:  "branch_time_in_parent": 109500.0,
amip_r1i1p1f1.json:  "branch_time_in_parent": 36500.0,
amip_r2i1p1f1.json:  "branch_time_in_parent": 73000.0,
amip_r3i1p1f1.json:  "branch_time_in_parent": 109500.0,
hist-aer_r1i1p1f1.json:  "branch_time_in_parent": 36500.0,
hist-aer_r2i1p1f1.json:  "branch_time_in_parent": 54750.0,
hist-aer_r3i1p1f1.json:  "branch_time_in_parent": 73000.0,
hist-aer_r4i1p1f1.json:  "branch_time_in_parent": 91250.0,
hist-aer_r5i1p1f1.json:  "branch_time_in_parent": 109500.0,
hist-GHG_r1i1p1f1.json:  "branch_time_in_parent": 36500.0,
hist-GHG_r2i1p1f1.json:  "branch_time_in_parent": 54750.0,
hist-GHG_r3i1p1f1.json:  "branch_time_in_parent": 73000.0,
hist-GHG_r4i1p1f1.json:  "branch_time_in_parent": 91250.0,
hist-GHG_r5i1p1f1.json:  "branch_time_in_parent": 109500.0,
historical_r1i1p1f1.json:  "branch_time_in_parent": 36500.0,
historical_r2i1p1f1.json:  "branch_time_in_parent": 54750.0,
historical_r3i1p1f1.json:  "branch_time_in_parent": 73000.0,
historical_r4i1p1f1.json:  "branch_time_in_parent": 91250.0,
historical_r5i1p1f1.json:  "branch_time_in_parent": 109500.0,
piClim-control_r1i1p1f1.json:  "branch_time_in_parent": 0.0,
piClim-histaer_r1i1p1f1.json:  "branch_time_in_parent": 7300.0,
piClim-histaer_r2i1p1f1.json:  "branch_time_in_parent": 10950.0,
piClim-histaer_r3i1p1f1.json:  "branch_time_in_parent": 14600.0,
piClim-histall_r1i1p1f1.json:  "branch_time_in_parent": 7300.0,
piClim-histall_r2i1p1f1.json:  "branch_time_in_parent": 10950.0,
piClim-histall_r3i1p1f1.json:  "branch_time_in_parent": 14600.0,
piControl_r1i1p1f1.json:  "branch_time_in_parent": 0.0,

We have NOT yet begun to process any of the hist-all-xGHG-xaer material. If these need exceptional metadata additions, we can accommodate.

[golaz]

@chengzhuzhang : no parent_experiment_id for amip is fine.

[chengzhuzhang]

@chengzhuzhang : no parent_experiment_id for amip is fine.

Thank you @golaz. As Tony noted, it will be some extra work (fixing + republish) to amend the metadata. I will try figure out if it is okay with cmor to have parent_experiment_id for amip for future runs.

[durack1]

@chengzhuzhang apologies for the slow uptake. This is an interesting one; the initialization of the land and river components is something that has not been specified for amip as it was always considered the atmosphere responded quickly to the boundary conditions, most often initialized from 1979 (even though the dataset provides temporal coverage from 1870), which is the requested start date for the CMIP: AMIP simulation (see here) and therefore initialization was less important. I believe that you could specify a parent_experiment_id and branch_time_in_parent using CMOR, these entries are not validated I believe, and would most often be parent_experiment_id = no parent and branch_time_in_parent = no parent. Pinging @taylor13 in case I have omitted any important details

[taylor13]

The AMIP experiment protocol called for simulations for the period 1979-2014. You have contributed a much longer run, which shouldn't really disrupt anything, but is non-standard. I can't remember if the checker checks that the parent_experiment_id is consistent with the CV specs. If it does, you'll have to omit parent_experiment_id or set it to "no parent". Similarly for some of the other "parent" global attributes that are text strings. For branch_time_in_parent, it should be omitted or set to 0.0d00.
I would note that there was an experiment called amip-hist called for by GMMIP, but it has not drawn much participation, and the SSTs prescribed for that run might be different from the AMIP SSTs.

[chengzhuzhang]

@durack1 @taylor13 Thank you both for your input! It turned out that cmor does checks for parent_experiment_id. After I add back parent_experiment_id, I ran into following error:

! Error: Your input attribute "parent_activity_id" with value
! "historical" needs to be replaced with value "no parent"
! as defined for experiment_id "amip".
!
!  See Control Vocabulary JSON file.(/home/zhang40/cmip6-cmor-tables/Tables//CMIP6_CV.json)

It explained why we took it out before publishing.

[durack1]

@chengzhuzhang thanks for the update, in some ways it's good that the checks dig down into this granularity, but on the alternate side it's not ideal if the "no parent" is not correct. To be honest, thinking about this, the land state would need to be initialized from something, and so "no parent" isn't really an ideal identifier. I would have assumed in a generic case, that all the DECK sims would be initialized from a piControl spinup, which aligns with the 1870 in your non-standard simulation. Thinking a little more, how would you initialize a ~1979 representative land state if you wanted the amip simulation initialization to be as close as possible to the boundary forcing? @golaz is there something for us (@taylor13) and I to learn here?

[chengzhuzhang]

@TonyB9000 Thank you for the updates. I added hist-nat json files that Chris asked for. I think this PR is ready to go. Please also go ahead to add five hist-nat simulations in datasm and publication.