Satellite Earth Radiation Budget Use Case : Comparison of OLR and OSR

giuliachierici commented 1 year ago

Notebook description

Hi, these notebooks are the preliminary (uncommented) versions for the comparison of different datasets regarding Earth Radiation Budget. They have the same structure, they only analyze different variables: the first analyzes the outgoing longwave radiation datasets (OLR), while the second the outgoing shortwave radiation datasets (OSR).

Notebook link or upload

satellite_userquestion_ERB.zip

Anything else we need to know?

I ran the notebooks having files stored locally and then I tried to run the API request over a small period of time (2-3 years) and it works. I'd like to run the notebooks for different periods for each satellite product (I've already tried but the API request failed):

2000-03, 2022-12 Period for NASA CERES
1979-01, 2022-12 Period for NOAA/NCEI HIRS
2017-01, 2021-12 Period for C3S CCI sent 3A
2018-01, 2021-12 Period for C3S CCI sent 3B
1995-01, 2003-12 Period for ESA CCI ERS2
2002-01, 2012-12 Period for ESA CCI ESA ENVISAT

Feel free to modify the code, I'm quite new to this.

Environment

malmans2 commented 1 year ago

Hi there,

FYI, I'm currently on leave and I'll be back on July 11th.

giuliachierici commented 1 year ago

Hi there, I uploaded a new version of the notebooks.

satellite_userquestion_ERB_01.zip

malmans2 commented 1 year ago

Hi @giuliachierici,

I just got back to work and I have a few items in my backlog. I'll try to take a look tomorrow though.

Do you have a deadline for this notebook?

giuliachierici commented 1 year ago

No problem! Nobody complained about that so I suppose that deadline wasn't so strict.

Il giorno mar 11 lug 2023 alle ore 18:28 Mattia Almansi < @.***> ha scritto:

Hi @giuliachierici https://github.com/giuliachierici,

I just got back to work and I have a few items in my backlog. I'll try to take a look tomorrow though.

Do you have a deadline for this notebook?

— Reply to this email directly, view it on GitHub https://github.com/bopen/c3s-eqc-toolbox-template/issues/77#issuecomment-1631131783, or unsubscribe https://github.com/notifications/unsubscribe-auth/BA53TY4XLDRSP4LCVDAWEOLXPV5JHANCNFSM6AAAAAAZYSZA3Q . You are receiving this because you were mentioned.Message ID: @.***>

malmans2 commented 1 year ago

Hi @giuliachierici,

The template for your use case is ready. You can find it here: https://github.com/bopen/c3s-eqc-toolbox-template/blob/main/notebooks/wp5/satellite_outgoing_radiation.ipynb.

I splitted ESA ESR because some data is missing in 2006.

There's a couple of parameters that you can use to customise the analysis: shortwave/longwave and region (e.g. northern/southern hemisphere).

I've already cached everything on the VM, so it should be quick if you run the notebook as it is. Here are the rendered notebooks I've executed on the VM: https://gist.github.com/malmans2/2e23ccf1e55c490171fb4ff5bb231958

Please let me know if everything is working as expected.

giuliachierici commented 1 year ago

Perfect! Everything works properly. I changed only the fact that ESA ESR was splitted (I've dropped the missing data). Thanks for the support!

vincenzodetoma commented 3 months ago

Hi @malmans2, I think I need some help also for this one attached here: satellite_earth_radiation_budget_outgoing_longwave_q01.zip Thanks, Vincenzo

malmans2 commented 3 months ago

Hi @vincenzodetoma,

Sorry about the delay. All the forms for this dataset have changed.

I need a new version of this dictionary:

variable = "longwave"
assert variable in ("longwave", "shortwave")

collection_id = "satellite-earth-radiation-budget"

request_dict = {
    "CERES": {
        "start": "2000-03",
        "stop": "2022-03",
        "format": "zip",
        "origin": "nasa_ceres_ebaf",
        "variable": f"outgoing_{variable}_radiation",
    },
    "Sentinel 3A": {
        "start": "2017-01",
        "stop": "2021-12",
        "format": "zip",
        "origin": "c3s_cci",
        "sensor": "slstr_on_sentinel_3a",
        "variable": "all_available_variables",
    },
    "Sentinel 3B": {
        "start": "2018-10",
        "stop": "2021-12",
        "format": "zip",
        "origin": "c3s_cci",
        "sensor": "slstr_on_sentinel_3b",
        "variable": "all_available_variables",
    },
    "ESA ENVISAT": {
        "start": "2002-05",
        "stop": "2012-04",
        "format": "zip",
        "origin": "esa_cloud_cci",
        "sensor": "aatsr",
        "variable": "all_available_variables",
    },
    "ESA ERS2": {
        "start": "1995-06",
        "stop": "2002-12",
        "format": "zip",
        "origin": "esa_cloud_cci",
        "sensor": "atsr2",
        "variable": "all_available_variables",
    },

    "HIRS": {
        "start": "1979-01",
        "stop": "2023-03",
        "format": "zip",
        "origin": "noaa_ncei_hirs",
        "variable": f"outgoing_{variable}_radiation",
    },
}

For each product you'd like to include in this analysis I need the start and stop of the time period, and all the parameters that define the product (with exception of the time parameters as we parametrise it with start and stop). Basically you need to go to the form page, and send me the latest version of the requests: https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-earth-radiation-budget?tab=form

Sorry but the forms changed quite a lot and I don't know much about these products. After you send me the new requests I will update the templates and fix the code if needed.

vincenzodetoma commented 3 months ago

ok, I will check with Andrea Storto and let you know! I tried to add him to these threads but I see it is not possible. Can we? Thanks!

malmans2 commented 3 months ago

This is a public repo, just tag him here using his GitHub username

vincenzodetoma commented 3 months ago

@andreastorto, like this?

vincenzodetoma commented 3 months ago

Hi @malmans2, here is the updated request according to last modifications in the CDS download tab:

variable = "longwave"
assert variable in ("longwave", "shortwave")

collection_id = "satellite-earth-radiation-budget"

request_dict = {
    "CERES": {
        "product_family": "ceres_ebaf",
        "climate_data_record_type":"thematic_climate_data_record",
        "time_aggregation":"monthly_mean",
        "start": "2000-03",
        "stop": "2024-02",
        "format": "zip",
        "origin": "nasa",
        "variable": f"outgoing_{variable}_radiation",
    },
    "Sentinel 3A": {
        "start": "2017-01",
        "stop": "2022-06",
        "format": "zip",
        "origin": "c3s",
        "sensor_on_satellite": "slstr_on_sentinel_3a",
        "variable": "all_variables",
        "product_family":"cci",
        "time_aggregation":"monthly_mean",
        "climate_data_record_type":"interim_climate_data_record",
    },
    "Sentinel 3B": {
        "start": "2018-10",
        "stop": "2022-06",
        "format": "zip",
        "origin": "c3s",
        "sensor_on_satellite": "slstr_on_sentinel_3b",
        "variable": "all_variables",
        "product_family":"cci",
        "time_aggregation":"monthly_mean",
        "climate_data_record_type":"interim_climate_data_record",
    },
    "Sentinel 3A_3B": {
        "start": "2018-10",
        "stop": "2022-06",
        "format": "zip",
        "origin": "c3s",
        "sensor_on_satellite": "slstr_on_sentinel_3a_3b",
        "variable": "all_variables",
        "product_family":"cci",
        "time_aggregation":"monthly_mean",
        "climate_data_record_type":"interim_climate_data_record",
    },
    "ESA ENVISAT": {
        "start": "2002-05",
        "stop": "2012-04",
        "format": "zip",
        "origin": "esa",
        "product_family": "cci",
        "climate_data_record_type":"thematic_climate_data_record",
        "time_aggregation": "monthly_mean",
        "sensor_on_satellite": "aatsr",
        "variable": "all_variables",
    },
    "ESA ERS2": {
        "start": "1995-06",
        "stop": "2003-01",
        "format": "zip",
        "origin": "esa",
        "product_family": "cci",
        "climate_data_record_type": "thematic_climate_data_record",
        "time_aggregation":"monthly_mean",
        "sensor_on_satellite": "atsr2",
        "variable": "all_variables",
    },
    "HIRS": {
        "start": "1979-01",
        "stop": "2024-04",
        "format": "zip",
        "origin": "noaa_ncei",
        "product_family": "hirs",
        "climate_data_record_type": "thematic_climate_data_record",
        "time_aggregation": "monthly_mean",
        "version": "2_7_reprocessed",
        "variable": f"outgoing_{variable}_radiation",
    },
   "CLARA_A3": {
      "start": "1979-01",
      "stop": "2020-12",
      "product_family": "clara_a3",
      "origin": "eumetsat",
      "variable" f"outgoing_{variable}_radiation",
      "climate_data_record_type": "thematic_climate_data_record",
      "time_aggregation": "monthly_mean",
   },

}

i.e., we have chosen to add everything new, in compliance with the old assessment. If any additional input is needed from us, please let us know! Thanks! @andreastorto

malmans2 commented 3 months ago

Hi @vincenzodetoma,

This template is also ready. I had to change the stop of ESA ERS2 from 2003-01 to 2002-12. I'm getting an "empty" file for January 2003, not sure if it's expected (to be precise, all variables are filled with 9E36).

Here is the template: https://github.com/bopen/c3s-eqc-toolbox-template/blob/main/notebooks/wp5/satellite_outgoing_radiation.ipynb

vincenzodetoma commented 3 months ago

Hi @malmans2, I've tested the notebook and all seems to be fine! Thanks a lot for your support! You can close the issue if you want!

bopen / c3s-eqc-toolbox-template