biothings / biothings_explorer_archived

BioThings Explorer: a schema-based client for API interoperability
Apache License 2.0
14 stars 14 forks source link

Support long-running API queries #142

Open andrewsu opened 3 years ago

andrewsu commented 3 years ago

Currently, BTE focuses on APIs that are generally microservices for knowledge retrieval. We will expand that scope to also include APIs for compute-intensive operations with longer run times. For example, Translator Reasoners, BigGIM, and/or the biclustering services may time out with a typical HTTP API call. Here, we will create a generic BTE wrapper system around long-running API calls, including asynchronous services that return results via a request ID. This system will allow a larger set of APIs to be a part of the BTE API ecosystem. As with all new features, we will create demo notebooks and documentation.

As one potential solution, the BTE wrapper could periodically poll the API for results while also sending the BTE client a "wait signal" that indicates the service is alive and still computing.