search

man-group / arctic

High performance datastore for time series and tick data
https://arctic.readthedocs.io/en/latest/
GNU Lesser General Public License v2.1
3.05k stars 583 forks source link

fix(list_symbols): #855 Use IXSCAN queries for the versions collection when querying symbols #856

Closed rob256 closed 3 years ago

rob256 commented 4 years ago

This is mainly a reversion on https://github.com/man-group/arctic/pull/520, but we add another index to avoid a FETCH stage which gives a massive speedup.

rob256 commented 4 years ago

Very nice - presumably this fixes quite a longstanding performance blooper?

Yeah. We moved from an IXSCAN+FETCH to a COLLSCAN, and this will allow us to do a straight IXSCAN. I've seen long queries on a collection which has a million documents, and this gives it a really nice speed up.

bmoscon commented 4 years ago

@rob256 any reason to not merge this?

rob256 commented 3 years ago

@rob256 any reason to not merge this?

Hi @bmoscon, I can't see a reason why not! I did re-run the checks but a lot of the tests are failing (similar to the failures on master).

I can update the changes.md once we're in a position to merge.

dunckerr commented 3 years ago

hi @bmoscon I'm working on this at Man AHL, and I was hoping to get this PR merged so I can go thru the release process. How do I get permissioned to work on this repo?

bmoscon commented 3 years ago

@dunckerr - if you work there I'm sure someone can add you to the org. @jamesblackburn can you help here?