biothings / biothings_explorer

TRAPI service for BioThings Explorer
https://api.bte.ncats.io
Apache License 2.0
8 stars 9 forks source link

Run all creative mode templates simultaneously #824

Open tokebe opened 1 week ago

tokebe commented 1 week ago

As a likely improved alternative to https://github.com/biothings/biothings_explorer/issues/808, we should instead try running all templates simultaneously.

This slightly complicates response merging, but race conditions should be solvable with a per-query execution lock.

More importantly, we still don't want to go over time, so we want to time out at about 4:55 to ensure there's time to finish up and make callbacks/store the response. This might be both on individual templates and on the entire handler.

After implementation, we'll want some performance testing to ensure this doesn't create unreasonable strain on memory/CPU.

rjawesome commented 1 week ago

See PRs

rjawesome commented 3 days ago

Note: It's likely that the current code for this PR will conflict with #825 because the code here aborts all subqueries when the time cap is reached. However, it is unclear how to handle this when the subqueries are in a queue seperate from the thread (and all subqueries from different queries are merged into one queue).

tokebe commented 3 days ago

Regarding integrating with #825, in that case we don't want to abort subqueries because we'd prefer them to keep running and cache results whether those results are being used immediately or not. So, if a template adds subqueries to the queue, and then the template execution is aborted, we'd want those subqueries to keep running -- it increases the chances the template will complete before timeout in subsequent queries to BTE.

tokebe commented 3 days ago

This being the case, after you've verified #825 performs well under high traffic, you might want to rebase branches addressing #825 to branches for this issue, so you can seamlessly merge the behavior of the two issues. Normally we wouldn't do this, but because both are very involved changes that will be going up simultaneously (and theoretically support one another, performance-wise), it makes sense to do so.