Reanalysis comparison with observations for validation purposes

cbartucca commented 1 week ago

Data Type

Reanalyses

Assessment Category

Model Performance

Dataset Name

reanalysis-era5-single-levels-monthly-means

Question Number

1

Workflow ID

eqctier3-d6e65ec8-fef8-4078-98a6-054a84916ea6

Zipped Notebook

reanalysis-era5-monthly-single-levels+reanalysis-era5-complete_validation_q01.zip

Environment

``` name: base channels: - defaults dependencies: - _libgcc_mutex=0.1=main - _openmp_mutex=5.1=1_gnu - boltons=23.0.0=py310h06a4308_0 - brotlipy=0.7.0=py310h7f8727e_1002 - bzip2=1.0.8=h7b6447c_0 - ca-certificates=2023.01.10=h06a4308_0 - certifi=2022.12.7=py310h06a4308_0 - cffi=1.15.1=py310h5eee18b_3 - charset-normalizer=2.0.4=pyhd3eb1b0_0 - conda=23.3.1=py310h06a4308_0 - conda-content-trust=0.1.3=py310h06a4308_0 - conda-package-handling=2.0.2=py310h06a4308_0 - conda-package-streaming=0.7.0=py310h06a4308_0 - cryptography=39.0.1=py310h9ce1e76_0 - idna=3.4=py310h06a4308_0 - jsonpatch=1.32=pyhd3eb1b0_0 - jsonpointer=2.1=pyhd3eb1b0_0 - ld_impl_linux-64=2.38=h1181459_1 - libffi=3.4.2=h6a678d5_6 - libgcc-ng=11.2.0=h1234567_1 - libgomp=11.2.0=h1234567_1 - libstdcxx-ng=11.2.0=h1234567_1 - libuuid=1.41.5=h5eee18b_0 - ncurses=6.4=h6a678d5_0 - openssl=1.1.1t=h7f8727e_0 - packaging=23.0=py310h06a4308_0 - pip=23.0.1=py310h06a4308_0 - pluggy=1.0.0=py310h06a4308_1 - pycosat=0.6.4=py310h5eee18b_0 - pycparser=2.21=pyhd3eb1b0_0 - pyopenssl=23.0.0=py310h06a4308_0 - pysocks=1.7.1=py310h06a4308_0 - python=3.10.10=h7a1cb2a_2 - readline=8.2=h5eee18b_0 - requests=2.28.1=py310h06a4308_1 - ruamel.yaml=0.17.21=py310h5eee18b_0 - ruamel.yaml.clib=0.2.6=py310h5eee18b_1 - setuptools=65.6.3=py310h06a4308_0 - six=1.16.0=pyhd3eb1b0_1 - sqlite=3.41.1=h5eee18b_0 - tk=8.6.12=h1ccaba5_0 - toolz=0.12.0=py310h06a4308_0 - tqdm=4.65.0=py310h2f386ee_0 - tzdata=2023c=h04d1e81_0 - urllib3=1.26.15=py310h06a4308_0 - wheel=0.38.4=py310h06a4308_0 - xz=5.2.10=h5eee18b_1 - zlib=1.2.13=h5eee18b_0 - zstandard=0.19.0=py310h5eee18b_0 prefix: /usr/local/miniconda3 ```

Anything else we need to know?

.

malmans2 commented 1 week ago

Hi @cbartucca,

There were a few issues I needed to fix:

You can have only one title (i.e., head level one with only one #). Methodology was also a title, so I changed it to ## Methodology
Do not use html syntax, only use markdown or myst syntax
You had a typo in your references (Thhe CDS...), but you should actually use myst syntax for references (the same you used for the sections)
Same for the bibliography reference with no URL, use the myst syntax

As I made a few changes, please make sure everything looks OK, and upload a new version if you'd like to make some modifications.

malmans2 commented 1 week ago

Oh wait, I just noticed you used the wrong syntax for all figures. You wrapped the figure object in md, so they are not rendered. I will fix that one as well and will share a new permalink.

malmans2 commented 1 week ago

A few other issues I found:

In the figure syntax, you must use : not =
Please use myst syntax for images as well

The full documentation is here: https://myst-parser.readthedocs.io/en/latest/syntax/images_and_figures.html

malmans2 commented 1 week ago

Permalink: https://github.com/ecmwf-projects/c3s2-eqc-quality-assessment/blob/f5b59a23cc0be52b3e5fa73eaae6a52b4eea3b67/Reanalyses/reanalysis_reanalysis-era5-single-levels-monthly-means_model-performance_q01.ipynb

Please ensure that the notebook name is correct. If you need to make any modifications (e.g., addressing the reviewer's comments), please share the new version of the notebook here, and I will provide a new permalink. Let me know once the notebook is accepted, and I will merge the PR into the main branch.

PREVIEW

cbartucca commented 1 week ago

Ciao Mattia, I replied to the email but maybe I should write here. I'm sorry you had to correct all these mistakes, I followed the template but without knowing anything about myst or html syntax. I can change the syntax I used for references, but I am not sure what you mean. The seasonal forecast notebook that was published on CDS and which I referred to in addition to the template, has the references in md. Here I attach a new version with a myst syntax for references, I hope I didn't mess up. The notebook name is correct. Thank you! reanalysis_reanalysis-era5-single-levels-monthly-means_model-performance_q01.zip

malmans2 commented 1 week ago

Hi Costanza,

This cell is the one that uses the figure syntax. The cell above shows the same content but wrapped with md, so the figure isn't rendered, allowing you to see the syntax directly. No worries, though—the important thing is that you've figured it out now.

However, there's another issue: it seems that all attachments were accidentally deleted while you were editing the notebook. If you download the last version you sent me, you'll notice that the figures aren't displayed.

cbartucca commented 1 week ago

Hi Mattia, it was my mistake, I zipped it outside the folder with the images. Let's try with this reanalysis_reanalysis-era5-single-levels-monthly-means_model-performance_q01.zip

malmans2 commented 1 week ago

Hi Costanza,

The attached images are still missing and you re-introduced a few of the errors I already fixed (e.g., you are using = instead of : in the figure syntax, and you got rid of the key-resources reference).

I suggest you start from the notebook I already fixed and you just edit that one (e.g., change figure settings, text, ...). If you don't know how to get the notebook, just click the permalink and then click the download button.

cbartucca commented 1 week ago

Yes I downloaded it but I thought it was better to modify the original by myself, to undestand errors. By the way this is the one you fixed, I just modified the syntax for the references reanalysis_reanalysis-era5-single-levels-monthly-means_model-performance_q01.zip

malmans2 commented 1 week ago

The last version you uploaded is still missing all attachments. Try downloading it and opening it, and you'll see that no figure is displayed.

Are you using jupyter installed on the VM? If no, my guess is that the software you are using to edit the notebook is deleting the attachments (probably some default settings). If that's the case, I suggest using jupyter on the VM as that's the only software we maintain and everything should be set up correctly.

cbartucca commented 1 week ago

I'm sorry, but by my side this is not a feasible check because figures were not displayed since the beginning, I only see some lines of code! Something like

widht: 700px height: 700px

And also, I'm not using your VM because it is a literature based notebook. The first one I attached and the subsequent edited versions were produced using the Jupyter server

malmans2 commented 1 week ago

If MyST is not rendered (you only see code in my version of the notebook), very likely you do not have the right software installed. You need to install jupyterlab-myst in order to display MyST.

That's why we use a VM with a common environment, so you don't have to deal with it. I suggest to try it out even if it's a literature based notebook. I can't help you much if you use your laptop as I don't have access to it.

cbartucca commented 1 week ago

Ok... again, I followed the guidance, which suggested to skip the vm part if you have a literature based notebook. I'm sorry, but please bear with me, this we are doing is a bit of a test we need internally to define the procedure. May I ask what the options would be at this point? Use the VM from the start, although it may be a bit overkill for a literature-based, or install the jupyterlab-myst, or? Perhaps insert images in a way other than drag and drop?

malmans2 commented 1 week ago

In my opinion the guidance should suggest to always use the VM (I think CNR is responsible for it, so you can discuss it with Chunxue/Vincenzo/Federico).

Of course evaluators can use their own resources, but I can't help much debugging.

On the VM, all you have to do is this:

ssh wp5@136.156.129.56
cd bartucca_constanza
jupyter_server
follow the instructions printed on terminal

That said, I'm pretty sure installing jupyterlab-myst on your machine would be enough to solve your issues (but I'm just guessing).

cbartucca commented 1 week ago

Of course, that's what I meant by internal testing, in the end hopefully we will have a clear workflow and template... access to the VM will take some time as I have to start with generating the keys. I'll get back to you as soon as I have a new notebook! Or new problems ahaha Thanks for the support!

malmans2 commented 1 week ago

OK, happy to help! If you want access to the VM, please send me your public key when you have one.

cbartucca commented 1 week ago

Meanwhile, actually the category 'Model Performance' in my case is not so correct. Initially it was 'Validation', that is one of the category listed in the google doc we (as evaluators) are referring to. I don't know if you are aware of this document, if not I share it here https://docs.google.com/document/d/15goKes3hmhinT17ch-xOUsU-PtVBUy_22kf0cXHp4cc/edit?usp=sharing I believe it could be useful to have a unique list!

malmans2 commented 1 week ago

Hi Costanza,

These are the assessment categories that have been communicated to me: https://github.com/ecmwf-projects/c3s2-eqc-quality-assessment?tab=readme-ov-file#naming-convention

I do have access to the document, but it appears to be a work in progress. Once you've finalized the names, please send me the exact list of categories you'd like to use. Please note that the assessment categories are used to build the names of the notebooks. Adding categories is perfectly fine, but removing or renaming them would be a breaking change, so the sooner the better.

cbartucca commented 1 week ago

I think this is something that Chris, and maybe Andre, should finalise. I think your list was sent to you by them, and at the same time on the Google document there are some comments from Chris like ‘add to Mattia's list’. I think, yes, categories like ‘validation’, ‘timeliness’ and ‘resolution’ should be added, while ‘mean’ I'm not sure makes sense. We will point this out to them and if you have the opportunity to do the same, that would be great.

malmans2 commented 1 week ago

Sounds good. The list was sent to me by CNR when Chris was not involved in this project yet. At the moment, the first meeting I have on my schedule with them is on Sept 17th.

fserva commented 1 week ago

Hi Costanza,

These are the assessment categories that have been communicated to me: https://github.com/ecmwf-projects/c3s2-eqc-quality-assessment?tab=readme-ov-file#naming-convention

I do have access to the document, but it appears to be a work in progress. Once you've finalized the names, please send me the exact list of categories you'd like to use. Please note that the assessment categories are used to build the names of the notebooks. Adding categories is perfectly fine, but removing or renaming them would be a breaking change, so the sooner the better.

Sorry to jump in - Mattia do you have a list of the categories already used for the latest version of the assessments? All those listed?

cbartucca commented 1 week ago

I see. Thank you, we will keep that date in mind!

malmans2 commented 1 week ago

@fserva I parsed all branches and this is what I've got:

{'climate-and-weather-extremes': {'climate_projections-cmip6_climate-and-weather-extremes_q01.ipynb',
                                  'climate_projections-cmip6_climate-and-weather-extremes_q02.ipynb',
                                  'climate_projections-cmip6_climate-and-weather-extremes_q03.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q01.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q02.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q03.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q04.ipynb',
                                  'insitu_insitu-gridded-observations-europe_climate-and-weather-extremes_q02.ipynb',
                                  'reanalysis_reanalysis-cerra-single-levels_climate-and-weather-extremes_q02.ipynb',
                                  'reanalysis_reanalysis-era5-pressure-levels_climate-and-weather-extremes_q03.ipynb',
                                  'satellite_satellite-fire-burned-area_climate-and-weather-extremes_q01.ipynb',
                                  'satellite_satellite-lake-water-level_climate-and-weather-extremes_q01.ipynb'},
 'climate-impact-indicators': {'climate_projections-cmip6_climate-impact-indicators_q01.ipynb',
                               'climate_projections-cmip6_climate-impact-indicators_q02.ipynb',
                               'climate_projections-cmip6_climate-impact-indicators_q03.ipynb',
                               'climate_projections-cmip6_climate-impact-indicators_q04.ipynb',
                               'satellite_satellite-sea-ice-concentration_climate-impact-indicators_q01.ipynb'},
 'climate-monitoring': {'insitu_insitu-gridded-observations-europe_climate-monitoring_q01.ipynb',
                        'insitu_insitu-gridded-observations-europe_climate-monitoring_q03.ipynb',
                        'insitu_insitu-gridded-observations-europe_climate-monitoring_q04.ipynb',
                        'reanalysis_reanalysis-carra-single-levels_climate-monitoring_q01.ipynb',
                        'reanalysis_reanalysis-era5-land-monthly-means_climate-monitoring_q01.ipynb',
                        'satellite_satellite-earth-radiation-budget_climate-monitoring_q01.ipynb',
                        'satellite_satellite-earth-radiation-budget_climate-monitoring_q02.ipynb',
                        'satellite_satellite-ozone-v1_climate-monitoring_q01.ipynb',
                        'satellite_satellite-sea-surface-temperature-ensemble-product_climate-monitoring_q01.ipynb',
                        'satellite_satellite-sea-surface-temperature_climate-monitoring_q01.ipynb',
                        'satellite_satellite-surface-radiation-budget_climate-monitoring_q01.ipynb'},
 'consistency-assessment': {'reanalysis_reanalysis-era5-pressure-levels-monthly-means_consistency-assessment_q01.ipynb',
                            'reanalysis_reanalysis-era5-pressure-levels-monthly-means_consistency-assessment_q02.ipynb',
                            'satellite_insitu-glaciers-extent_consistency-assessment_q02.ipynb',
                            'satellite_satellite-aerosol-properties_consistency-assessment_q01.ipynb',
                            'satellite_satellite-aerosol-properties_consistency-assessment_q02.ipynb',
                            'satellite_satellite-aerosol-properties_consistency-assessment_q03.ipynb',
                            'satellite_satellite-albedo_consistency-assessment_q01.ipynb',
                            'satellite_satellite-albedo_consistency-assessment_q02.ipynb',
                            'satellite_satellite-albedo_consistency-assessment_q03.ipynb',
                            'satellite_satellite-ice-sheet-elevation-change_consistency-assessment_q02.ipynb',
                            'satellite_satellite-land-cover_consistency-assessment_q01.ipynb',
                            'satellite_satellite-ocean-colour_consistency-assessment_q01.ipynb',
                            'satellite_satellite-total-column-water-vapour-land-ocean_consistency-assessment_q01.ipynb',
                            'satellite_satellite-upper-troposphere-humidity_consistency-assessment_q01.ipynb'},
 'data-completeness': {'satellite_derived-gridded-glacier-mass-change_data-completeness_q01.ipynb',
                       'satellite_satellite-lake-water-temperature_data-completeness_q01.ipynb',
                       'satellite_satellite-soil-moisture_data-completeness_q01.ipynb'},
 'forecast-skill': {'seasonal_seasonal-monthly-single-levels_forecast-skill_q02.ipynb',
                    'seasonal_seasonal-monthly-single-levels_forecast-skill_q04.ipynb',
                    'seasonal_seasonal-monthly-single-levels_forecast-skill_q99.ipynb'},
 'intercomparison': {'satellite_satellite-sea-ice-concentration_intercomparison_q02.ipynb',
                     'satellite_satellite-sea-ice-concentration_intercomparison_q03.ipynb',
                     'satellite_satellite-sea-ice-edge-type_intercomparison_q01.ipynb'},
 'model-performance': {'climate_projections-cmip6_model-performance_q02.ipynb',
                       'climate_projections-cmip6_model-performance_q03.ipynb',
                       'reanalysis_reanalysis-era5-single-levels-monthly-means_model-performance_q01.ipynb'},
 'trend-assessment': {'reanalysis_reanalysis-cerra-single-levels_trend-assessment_q02.ipynb',
                      'satellite_derived-gridded-glacier-mass-change_trend-assessment_q02.ipynb',
                      'satellite_satellite-carbon-dioxide_trend-assessment_q01.ipynb',
                      'satellite_satellite-fire-burned-area_trend-assessment_q02.ipynb',
                      'satellite_satellite-greenland-ice-sheet-velocity_trend-assessment_q01.ipynb',
                      'satellite_satellite-ice-sheet-elevation-change_trend-assessment_q01.ipynb',
                      'satellite_satellite-ice-sheet-mass-balance_trend-assessment_q02.ipynb',
                      'satellite_satellite-land-cover_trend-assessment_q02.ipynb',
                      'satellite_satellite-sea-ice-thickness_trend-assessment_q01.ipynb',
                      'satellite_satellite-sea-surface-temperature-ensemble-product_trend-assessment_q02.ipynb',
                      'satellite_satellite-sea-surface-temperature_trend-assessment_q02.ipynb'},
 'uncertainty': {'satellite_insitu-glaciers-extent_uncertainty_q01.ipynb',
                 'satellite_satellite-greenland-ice-sheet-velocity_uncertainty_q02.ipynb',
                 'satellite_satellite-humidity-profiles_uncertainty_q01.ipynb',
                 'satellite_satellite-ice-sheet-mass-balance_uncertainty_q01.ipynb',
                 'satellite_satellite-methane_uncertainty_q01.ipynb',
                 'satellite_satellite-precipitation_uncertainty_q01.ipynb'},
 'variability': {'insitu_insitu-observations-gruan-reference-network_variability_q01.ipynb',
                 'reanalysis_reanalysis-era5-pressure-levels-monthly-means_variability_q04.ipynb'}}

fserva commented 1 week ago

@fserva I parsed all branches and this is what I've got:

{'climate-and-weather-extremes': {'climate_projections-cmip6_climate-and-weather-extremes_q01.ipynb',
                                  'climate_projections-cmip6_climate-and-weather-extremes_q02.ipynb',
                                  'climate_projections-cmip6_climate-and-weather-extremes_q03.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q01.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q02.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q03.ipynb',
                                  'climate_projections-cordex-domains-single-levels_climate-and-weather-extremes_q04.ipynb',
                                  'insitu_insitu-gridded-observations-europe_climate-and-weather-extremes_q02.ipynb',
                                  'reanalysis_reanalysis-cerra-single-levels_climate-and-weather-extremes_q02.ipynb',
                                  'reanalysis_reanalysis-era5-pressure-levels_climate-and-weather-extremes_q03.ipynb',
                                  'satellite_satellite-fire-burned-area_climate-and-weather-extremes_q01.ipynb',
                                  'satellite_satellite-lake-water-level_climate-and-weather-extremes_q01.ipynb'},
 'climate-impact-indicators': {'climate_projections-cmip6_climate-impact-indicators_q01.ipynb',
                               'climate_projections-cmip6_climate-impact-indicators_q02.ipynb',
                               'climate_projections-cmip6_climate-impact-indicators_q03.ipynb',
                               'climate_projections-cmip6_climate-impact-indicators_q04.ipynb',
                               'satellite_satellite-sea-ice-concentration_climate-impact-indicators_q01.ipynb'},
 'climate-monitoring': {'insitu_insitu-gridded-observations-europe_climate-monitoring_q01.ipynb',
                        'insitu_insitu-gridded-observations-europe_climate-monitoring_q03.ipynb',
                        'insitu_insitu-gridded-observations-europe_climate-monitoring_q04.ipynb',
                        'reanalysis_reanalysis-carra-single-levels_climate-monitoring_q01.ipynb',
                        'reanalysis_reanalysis-era5-land-monthly-means_climate-monitoring_q01.ipynb',
                        'satellite_satellite-earth-radiation-budget_climate-monitoring_q01.ipynb',
                        'satellite_satellite-earth-radiation-budget_climate-monitoring_q02.ipynb',
                        'satellite_satellite-ozone-v1_climate-monitoring_q01.ipynb',
                        'satellite_satellite-sea-surface-temperature-ensemble-product_climate-monitoring_q01.ipynb',
                        'satellite_satellite-sea-surface-temperature_climate-monitoring_q01.ipynb',
                        'satellite_satellite-surface-radiation-budget_climate-monitoring_q01.ipynb'},
 'consistency-assessment': {'reanalysis_reanalysis-era5-pressure-levels-monthly-means_consistency-assessment_q01.ipynb',
                            'reanalysis_reanalysis-era5-pressure-levels-monthly-means_consistency-assessment_q02.ipynb',
                            'satellite_insitu-glaciers-extent_consistency-assessment_q02.ipynb',
                            'satellite_satellite-aerosol-properties_consistency-assessment_q01.ipynb',
                            'satellite_satellite-aerosol-properties_consistency-assessment_q02.ipynb',
                            'satellite_satellite-aerosol-properties_consistency-assessment_q03.ipynb',
                            'satellite_satellite-albedo_consistency-assessment_q01.ipynb',
                            'satellite_satellite-albedo_consistency-assessment_q02.ipynb',
                            'satellite_satellite-albedo_consistency-assessment_q03.ipynb',
                            'satellite_satellite-ice-sheet-elevation-change_consistency-assessment_q02.ipynb',
                            'satellite_satellite-land-cover_consistency-assessment_q01.ipynb',
                            'satellite_satellite-ocean-colour_consistency-assessment_q01.ipynb',
                            'satellite_satellite-total-column-water-vapour-land-ocean_consistency-assessment_q01.ipynb',
                            'satellite_satellite-upper-troposphere-humidity_consistency-assessment_q01.ipynb'},
 'data-completeness': {'satellite_derived-gridded-glacier-mass-change_data-completeness_q01.ipynb',
                       'satellite_satellite-lake-water-temperature_data-completeness_q01.ipynb',
                       'satellite_satellite-soil-moisture_data-completeness_q01.ipynb'},
 'forecast-skill': {'seasonal_seasonal-monthly-single-levels_forecast-skill_q02.ipynb',
                    'seasonal_seasonal-monthly-single-levels_forecast-skill_q04.ipynb',
                    'seasonal_seasonal-monthly-single-levels_forecast-skill_q99.ipynb'},
 'intercomparison': {'satellite_satellite-sea-ice-concentration_intercomparison_q02.ipynb',
                     'satellite_satellite-sea-ice-concentration_intercomparison_q03.ipynb',
                     'satellite_satellite-sea-ice-edge-type_intercomparison_q01.ipynb'},
 'model-performance': {'climate_projections-cmip6_model-performance_q02.ipynb',
                       'climate_projections-cmip6_model-performance_q03.ipynb',
                       'reanalysis_reanalysis-era5-single-levels-monthly-means_model-performance_q01.ipynb'},
 'trend-assessment': {'reanalysis_reanalysis-cerra-single-levels_trend-assessment_q02.ipynb',
                      'satellite_derived-gridded-glacier-mass-change_trend-assessment_q02.ipynb',
                      'satellite_satellite-carbon-dioxide_trend-assessment_q01.ipynb',
                      'satellite_satellite-fire-burned-area_trend-assessment_q02.ipynb',
                      'satellite_satellite-greenland-ice-sheet-velocity_trend-assessment_q01.ipynb',
                      'satellite_satellite-ice-sheet-elevation-change_trend-assessment_q01.ipynb',
                      'satellite_satellite-ice-sheet-mass-balance_trend-assessment_q02.ipynb',
                      'satellite_satellite-land-cover_trend-assessment_q02.ipynb',
                      'satellite_satellite-sea-ice-thickness_trend-assessment_q01.ipynb',
                      'satellite_satellite-sea-surface-temperature-ensemble-product_trend-assessment_q02.ipynb',
                      'satellite_satellite-sea-surface-temperature_trend-assessment_q02.ipynb'},
 'uncertainty': {'satellite_insitu-glaciers-extent_uncertainty_q01.ipynb',
                 'satellite_satellite-greenland-ice-sheet-velocity_uncertainty_q02.ipynb',
                 'satellite_satellite-humidity-profiles_uncertainty_q01.ipynb',
                 'satellite_satellite-ice-sheet-mass-balance_uncertainty_q01.ipynb',
                 'satellite_satellite-methane_uncertainty_q01.ipynb',
                 'satellite_satellite-precipitation_uncertainty_q01.ipynb'},
 'variability': {'insitu_insitu-observations-gruan-reference-network_variability_q01.ipynb',
                 'reanalysis_reanalysis-era5-pressure-levels-monthly-means_variability_q04.ipynb'}}

Thanks @malmans2, very interesting. How complicated would be to change the categories, in case? Could you do it programmatically on your side or do we need to involve evaluators, in case?

FYI @vincenzodetoma

malmans2 commented 1 week ago

I can handle this programmatically, but the evaluators will need to re-submit the permalink. Changing the filename will also alter the Jupyter Book URL, which is automatically inferred in the CIM from the permalink. Additionally, if you change the filename of notebooks that have already been pushed to production, someone will need to update the Jupyter Book link displayed on the EQC page.

fserva commented 1 week ago

I can handle this programmatically, but the evaluators will need to re-submit the permalink. Changing the filename will also alter the Jupyter Book URL, which is automatically inferred in the CIM from the permalink. Additionally, if you change the filename of notebooks that have already been pushed to production, someone will need to update the Jupyter Book link displayed on the EQC page.

Good, how many have been already pushed to production, if you know that? I think only seasonal forecast was published so far, so perhaps 2 of them.

malmans2 commented 1 week ago

Yes, but I think there's more that have been approved and just need to be merged. Before you make any decisions, make sure Chris is OK with it. Last time we talked about this he mentioned changes that would not impact the notebooks that are already under review.

malmans2 commented 6 days ago

FYI, I added a few categories as requested in #169

ecmwf-projects / c3s2-eqc-quality-assessment