search

blaylockbk / goes2go

Download and process GOES-16 and GOES-17 data from NOAA's archive on AWS using Python.
https://goes2go.readthedocs.io/
MIT License
205 stars 38 forks source link

Fix for FileNotFoundError when there is no GOES data. #106

Closed ifscript closed 1 month ago

ifscript commented 1 month ago

Hi

I've coded an option to ignore FileNotFoundError errors raised by the S3FileSystem.ls() function. The error occurs because the goes2go library assumes to find a folder for each hour of the day, but due to satellite outages or similar events, some folders are missing.

This fixes issue #93 and addresses the discussion #86

This issue has bothered me several times and I hope my fix is up to standard. If not, I'd love it if you implement something similar.

blaylockbk commented 1 month ago

Thanks for the PR. Looks great. I like that you added the option to the default config file, too.

I'll merge this now, but do you have a specific date that was problematic? It would be nice to eventually add a test for this case.

ifscript commented 1 month ago

Hi, thanks for merging.

One example would be the 6th January 2020, which is also mentioned in #86.

For the summer months JJA 2020 to 2022 the days with missing data are:

The latter two dates are specially interesting, because only a part of the channels are missing, mostly during 17-18 UTC. This leads to missing data in the L2 products.