Open hexylena opened 6 years ago
I agree with this. We should probably do this when refactoring shed-tools #91 . The code is very spaghetti ish now. I volunteer, but it is unclear yet when I have time to do this. Probably somewhere in july august.
I have been thinking about this. Wouldn't be a global BIOBLEND_RETRY environment variable or some other setting in bioblend preferable? I mean, it would be a lot more effective than fixing bioblend's errors in all the tools that use it. What do you think about this @erasche ?
Yeah that could work @rhpvorderman. Something sort of like logging?
import bioblend
bioblend.clientConfig(retry=5)
You have a good point that it'd be easier to fix there than everything that depends on it. @nsoranzo do you have an opinion on this?
For GET requests it could be added to BioBlend, for POST probably not a good idea. Not sure about PUT and DELETE.
obviously will default to 0 so we don't add a footgun for bioblend users.
retry_dangerous=False
in client config?For a start we can add a retry_idempotent
, i.e. excluding POST and forcing it to 0 for the quota PUT requests.
@rhpvorderman you can do some of this already via
from bioblend import galaxy
gx = galaxy.GalaxyInstance(...)`
gx.max_get_attempts(5)
@nsoranzo I'm working up a PR now. Would allowing the user to choose which methods to retry be desirable? retry_idempotent
sounds good though.
I think users may not be too familiar with REST API concepts to make the best choice, depends if there is a use case.
How about something like this? (Does not have to be this) https://github.com/galaxyproject/bioblend/pull/266 Has retry_idempotent
so users can use that.
I think my only case is things like installation, but there I guess we possibly need more complex/intelligent retry logic.
We've been running our automated tool installation and we occasionally see errors due to transient network failures that cause failures of the jenkins job. If we wrapped all bioblend / external network connections in a retry (we use https://github.com/litl/backoff for some stuff here) this would be extremely helpful.
E.g.