search

biothings / biothings_explorer_archived

BioThings Explorer: a schema-based client for API interoperability
Apache License 2.0
14 stars 14 forks source link

retry after failed API call #49

Open andrewsu opened 4 years ago

andrewsu commented 4 years ago

This is a snippet from https://github.com/biothings/biothings_explorer/blob/master/jupyter%20notebooks/TIDBIT%2002%20Finding%20New%20Uses%20for%20Existing%20Drugs%20to%20Treat%20Parkinson%E2%80%99s%20Disease.ipynb QUERY 2.1 Step 2, 

API 3.13: http://www.dgidb.org/api/v2/interactions.json?genes=RPL14
API 3.12: http://www.dgidb.org/api/v2/interactions.json?genes=SKP1
API 3.15: http://www.dgidb.org/api/v2/interactions.json?genes=SCG3
API 3.9: http://www.dgidb.org/api/v2/interactions.json?genes=VCP
dgidb_gene2chemical failed
API 3.67: http://www.dgidb.org/api/v2/interactions.json?genes=SPTSSB
dgidb_gene2chemical failed
API 3.68: http://www.dgidb.org/api/v2/interactions.json?genes=LTB

From slack, more details here:

kevinxin 1 hour ago dgidb couldn’t handle large amounts of concurrent queries (if we send like 10 requests at the same time, some will fail).

andrew 1 hour ago So the server returns a 500 error (or similar)? And in that scenario, does BTE retry that query, or is it skipped?

kevinxin 1 hour ago It’s skipped at the moment. The retry mechanism hasn’t been implemented yet.

We should implement a retry mechanism, as suggested...

PaulBryden commented 4 years ago

Hi, I'm Paul, a Masters Graduate from Strathclyde University in Scotland. I think this project is very exciting and the power that it will bring to genomic data analytics and machine learning is huge. Please see my first pull request which provides a retry mechanism for API calls and fixes the issue identified inside the Treat Parkinson's ipynb. Thanks, Paul Bryden