search

serratus-bio / open-virome

monorepo for data explorer UI and APIs
http://openvirome.com/
GNU Affero General Public License v3.0
0 stars 0 forks source link

queryBuilder aggregator improvement #52

Closed almosnow closed 3 months ago

almosnow commented 3 months ago

Hi Luke,

There's this data structure that keeps track of the IDs that are requested and creates ranges of them.

I've noticed many of these ranges are just two consecutive IDs, e.g. "PR0002", "PR0003" gets turned into RANGE("PR0002"-"PR0003"). I feel ranges should be three or more, otherwise there's some overhead that we could avoid.

One example where this matters, it would an hypothetical query from:

SELECT * FROM table WHERE id IN ('PR0002', 'PR0003', 'PR0005', 'PR0006')

to:

SELECT * FROM table WHERE id BETWEEN 'PR0002' AND 'PR0003' AND BETWEEN 'PR0005' AND 'PR0006')

which kind of defeats the purpose of it.

I'm sure the change is trivial but I haven't read your code, so not familiar.

lukepereira commented 3 months ago

Yes, i noticed this as well but haven't gotten around to fixing. Just need to increase the range by one to prevent it from happening, will update soon

almosnow commented 3 months ago

Yeah, should be a super trivial change, but idk where and didn't want to mess up your code, lol.

lukepereira commented 3 months ago

We can close this since we'll be disabling the BETWEEN SQL queries. We likely want add back the reduced ranges in the frontend, but I'll open a separate task for that