Update AERONET data version

ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP

https://www.esmvaltool.org

Apache License 2.0

210 stars 121 forks source link

Update AERONET data version #3692

Open ehogan opened 5 days ago

ehogan commented 5 days ago

I wanted to highlight a question I asked on the issue: I'm not sure whether the dates in the CMORizers needs updating?

Description

Closes #3690
Link to documentation:
- *

Before you get started

[x] ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[x] 🛠 Code is written according to the code quality guidelines
[x] 🛠 Tests run successfully
[x] 🛠 All checks below this pull request were successful

New or updated recipe/diagnostic

[x] 🧪 Recipe runs successfully
[x] 🧪 Figure(s) and data look as expected from literature

To help with the number of pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

bouweandela commented 5 days ago

I wanted to highlight a question I asked on the issue: I'm not sure whether the dates in the CMORizers needs updating?

@ESMValGroup/obs-maintainers Does anyone know the answer to this question?

valeriupredoi commented 5 days ago

I must confess I am not in the know, sorry - I only reviewed all that code from a technical side, maybe @catherinehardacre and/or @alistairsellar know, aside from the OBS team? :beer:

catherinehardacre commented 2 days ago

Hi @ehogan,

I'm happy to approve the date change in recipe_aod_aeronet_assess.yml so that it aligns with the DKRZ data. Thank you for tidying this up for us.

Regarding the cmorizer - I think you mean the date in https://github.com/ESMValGroup/ESMValTool/blob/main/esmvaltool/cmorizers/data/formatters/datasets/aeronet.py? (Which refers to the date when the Aeronet data was last accessed.) Technically for the code to run, this does not need to be changed. However, if the ESMValTool community would like everything to be aligned/look tidy, then this could be changed to 20240406.

Will the date in recipe_aod_aeronet_assess.yml (and therefore aeronet.py) need to changed for every release? If so this is not something I'd considered in the design... Are there similar requirements for other datasets?

Thanks, Catherine

ehogan commented 1 day ago

Hi @ehogan,

Hi Catherine! 😁

I'm happy to approve the date change in recipe_aod_aeronet_assess.yml so that it aligns with the DKRZ data. Thank you for tidying this up for us.

Super, thank you!

Regarding the cmorizer - I think you mean the date in https://github.com/ESMValGroup/ESMValTool/blob/main/esmvaltool/cmorizers/data/formatters/datasets/aeronet.py? (Which refers to the date when the Aeronet data was last accessed.) Technically for the code to run, this does not need to be changed. However, if the ESMValTool community would like everything to be aligned/look tidy, then this could be changed to 20240406.

There are other dates in esmvaltool/cmorizers/data/cmor_config/AERONET.yml and esmvaltool/cmorizers/data/datasets.yml too?

Will the date in recipe_aod_aeronet_assess.yml (and therefore aeronet.py) need to changed for every release? If so this is not something I'd considered in the design... Are there similar requirements for other datasets?

I don't think so, but would @ESMValGroup/obs-maintainers or @ESMValGroup/technical-lead-development-team be able to clarify, please? 😊

catherinehardacre commented 1 day ago

@ehogan - oops, I missed the other dates in https://github.com/ESMValGroup/ESMValTool/blob/baa34009a6332e9e3ca4c9e2a09e385394dca1e2/esmvaltool/cmorizers/data/cmor_config/AERONET.yml and https://github.com/ESMValGroup/ESMValTool/blob/baa34009a6332e9e3ca4c9e2a09e385394dca1e2/esmvaltool/cmorizers/data/datasets.yml.

I think these should all be changed to align with the DKRZ data, i.e. changed to 20240406. While the date in datasets.yml is just indicating that the data set has been recently accessed and should be 'live' (as per aeronet.py), in AERONET.yml , the date feeds into the formatted dataset and I think this should align with the DKRZ data.

So, for clarity and consistency, I think all dates should be the same. I hope this does not hold thing up for the release. I'm happy to pre-emptively approve this (but can also take a look tomorrow).

Thanks @ehogan :)

ehogan commented 17 hours ago

@ehogan - oops, I missed the other dates in https://github.com/ESMValGroup/ESMValTool/blob/baa34009a6332e9e3ca4c9e2a09e385394dca1e2/esmvaltool/cmorizers/data/cmor_config/AERONET.yml and https://github.com/ESMValGroup/ESMValTool/blob/baa34009a6332e9e3ca4c9e2a09e385394dca1e2/esmvaltool/cmorizers/data/datasets.yml.

I think these should all be changed to align with the DKRZ data, i.e. changed to 20240406. While the date in datasets.yml is just indicating that the data set has been recently accessed and should be 'live' (as per aeronet.py), in AERONET.yml , the date feeds into the formatted dataset and I think this should align with the DKRZ data.

So, for clarity and consistency, I think all dates should be the same. I hope this does not hold thing up for the release. I'm happy to pre-emptively approve this (but can also take a look tomorrow).

Thanks @ehogan :)

Thanks Catherine! All done in db59f2600 😊