spaceml-org / GIBS-Downloader

GIBS Downloader is a command-line tool which facilitates the downloading of NASA satellite imagery and offers different functionalities in order to prepare the images for training in a machine learning pipeline.
http://spaceml.org/repo/project/605b7b751644770011e850c3/false/
Apache License 2.0
44 stars 8 forks source link

Add error handling for faulty downloads #20

Open fernandolis10 opened 3 years ago

fernandolis10 commented 3 years ago

For all commands that depend on the original image being downloaded properly (e.g. tiling, writing to tfrecords, generating video), implement some sort of check that the original images we are using are what we expect them to be. Things to consider:

Alternatively, instead of implementing the check while tilling or writing to tfrecords AFTER the images are downloaded, catch any errors from the call to gdal_translate which may signal that the data being downloaded is faulty WHILE it is downloading.

Unexpected errors/warnings that have sometimes occurred in downloading:

mcechini commented 3 years ago

I think we'd be interested in knowing what requests failed and when the occurred. That way we can see if there is a server/service issue on our end.

fernandolis10 commented 3 years ago

I think we'd be interested in knowing what requests failed and when the occurred. That way we can see if there is a server/service issue on our end.

There was a request today (3/3/21) for an image of the whole world, which should have occurred between 10am-12:30pm est which resulted in the timeout error