neurobagel / api

https://api.neurobagel.org/
MIT License
4 stars 4 forks source link

Improve inefficient SPARQL query template #307

Closed surchs closed 5 months ago

surchs commented 5 months ago

Our current SPARQL query template has a couple of problems that make it slow:

Unnecessary nesting of OPTIONAL clauses https://github.com/neurobagel/api/blob/1e9ef34d54b89b824c42f1e6293a0ab9d5931c65/app/api/utility.py#L232-L237

and

https://github.com/neurobagel/api/blob/1e9ef34d54b89b824c42f1e6293a0ab9d5931c65/app/api/utility.py#L258-L264 (even worse)

in order of importance

also: https://github.com/neurobagel/api/blob/1e9ef34d54b89b824c42f1e6293a0ab9d5931c65/app/api/utility.py#L207-L211

is most likely not necessary - it would only help to capture those subjects who do not have any file system path associated and thus would not be datalad gettable.

TODOs:

From initial testing, this will let us cut query execution time by an order of magnitude.

See also: https://github.com/neurobagel/planning/issues/142

surchs commented 5 months ago

Turns out the main thing was really just stating explicitly what session type we want for the imaging session. That cut the query execution time by ~ 90%

surchs commented 5 months ago

:rocket: Issue was released in v0.2.1 :rocket: