vandersat / vds-api-client

API client for the VanderSat API
MIT License
3 stars 18 forks source link

Question on retry for gridded data requests #45

Closed sshoaie closed 3 years ago

sshoaie commented 3 years ago

I have a question from a user of our API:

"I would like to ask you about the response time in API. On Saturday we requested 3 days (2. – 4.6.) and for 2 of them (2., 3.) we did not receive any response for 5 minutes (any change of prepared data percentage), for the third one, it was very near 5 minutes when we got the response. We have in our code a timeout limit, after which we stop requesting the files and log a warning message that the data were not downloaded. What is the maximum timeout time we should consider? Or should we rather try to download files a bit later, if we dont get any response in 5 minutes and dates are included in available dates?"

I should note that this user is requesting gridded data.

I see that in the code of the API we already use the @retry decorator for the following methods:

but that we do not use it for the gen_gridded_data_request.

Is my understanding correct that if they submit something similarly as this:

vds.set_outfold(os.path.join(root, 'point-time-series'))
vds.gen_gridded_data_request(products=product, start_date=start_date, end_date=end_date, lat_min=lat_min, lat_max=lat_max, lon_min=lon_min, lon_max=lon_max)

vds.submit_async_requests()
vds.download_async_files()

that the retry decorators are already used? Or, how should the question be addressed?

cpaulik commented 3 years ago

I don't think this is an issue of the vds_api_client library but rather about how our async API requests work.

The docs should explain it. If they are unclear then feel free to raise an issue in the internal project / or adapt them so the user gets the information they need.

closing this / reopen if needed.