EcoExtreML / STEMMUS_SCOPE_Processing

Python modules and notebooks to process data and run STEMMUS-SCOPE model
https://pystemmusscope.readthedocs.io/en/latest/
Apache License 2.0
5 stars 1 forks source link

Add LAI to global data selection #69

Closed BSchilperoort closed 1 year ago

BSchilperoort commented 1 year ago

This PR adds LAI data to the global data selection. LAI comes from the fapar dataset, downloaded using the CDS. (The CDS fapar data has download issues, but this is under investigation by ECMWF).

Currently LAI data for 2014 is on Snellius. (same as ERA5 and ERA5-land).

Tasks:

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

geek-yang commented 1 year ago

Got an unexpected error when running the download_FAPAR_LAI.py script. Any idea about it? It doesn't seem to be an error related to this PR 😿 .

image

BSchilperoort commented 1 year ago

Got an unexpected error when running the download_FAPAR_LAI.py script. Any idea about it? It doesn't seem to be an error related to this PR 😿 .

There might be a typo in there, while I trying to make a working script. However, the fapar dataset has issues on the CDS, so you can probably skip that part for now until ECMWF/Vito fixes this.

The downloads are slow and unstable due to the data not being hosted on the ECWMWF's servers, but instead on a Vito server.

When you make a request using the CDS api, the CDS backend downloads the files from the vito server, and once they're downloaded send them to you. However, the connection between the CDS server and the Vito server is not stable and can cause the download to fail or be corrupted.

I have raised this issue on the CDS api github repo, and opened a ticket on the ECMWF website.