openclimatefix / Satip

Satip contains the code necessary for retrieving, transforming and storing EUMETSAT data

https://satip.readthedocs.io/

MIT License

41 stars 28 forks source link

Extend GOES data retrieval to include NOAA's CLAss archive #249

Closed 14Richa closed 5 months ago

14Richa commented 6 months ago

Pull Request

Description

This PR adds a new function download_archival_goes_data to the GOESDownloadManager class. This function allows users to download archival GOES data from NOAA's CLASS archive.

Checklist:

[x] My code follows OCF's coding style guidelines
[x] I have performed a self-review of my own code
[x] I have checked my code and corrected any misspellings

14Richa commented 6 months ago

Hey @jacobbieker, as discussed, I have extended the GOESDownloadManager with 'NOAS' for archival data. Please review when you get time.

jacobbieker commented 6 months ago

Hi, thanks for adding this! I might have misspoke with the archival data that I meant. From CLAss we want to be able to download the GOES-8 to GOES-15 data (available here: https://www.aev.class.noaa.gov/saa/products/search?datatype_family=GVAR_IMG). This might also work for that, but not sure, could you check?

14Richa commented 5 months ago

Hi, thanks for adding this! I might have misspoke with the archival data that I meant. From CLAss we want to be able to download the GOES-8 to GOES-15 data (available here: https://www.aev.class.noaa.gov/saa/products/search?datatype_family=GVAR_IMG). This might also work for that, but not sure, could you check?

Hey @jacobbieker, Thank you for clarifying the requirements.

I've updated the code to check the provided URL for the availability of GOES-8 to GOES-15 data before proceeding with the download. If the data is found, it will be downloaded for each satellite in the specified range.

jacobbieker commented 5 months ago

Great! Thank you. Have you tested this? And could you add a test for downloading GOES-15 data from CLAss? And have it actually download the file too, as it shouldn't be that large.

14Richa commented 5 months ago

Great! Thank you. Have you tested this? And could you add a test for downloading GOES-15 data from CLAss? And have it actually download the file too, as it shouldn't be that large.

Yes, I've tested it locally, and it is working fine. I've also added a unit test for it.

jacobbieker commented 5 months ago

Okay, great! Just closing and reopening to trigger our CI, once these pass, should be almost good to go!

jacobbieker commented 5 months ago

Hi, I'm not sure this actually does what we want unfortunately, it downloads a HTML file, but not the NetCDF file from GOES-15 from CLAss, so we might want to do some more work on this!

14Richa commented 5 months ago

Hi, I'm not sure this actually does what we want unfortunately, it downloads a HTML file, but not the NetCDF file from GOES-15 from CLAss, so we might want to do some more work on this!

Hi @jacobbieker , I am working on it.