search

blevesearch / zapx

Zap file format compatible with a future version of Bleve
Apache License 2.0
11 stars 12 forks source link

MB-60943 - Change quantiser when vector index is tuned for memory #244

Closed metonymic-smokey closed 4 months ago

metonymic-smokey commented 4 months ago

This PR changes the quantiser to SQ4 from SQ8 when the index is optimised for memory.

abhinavdangeti commented 4 months ago

@metonymic-smokey did you have a document of your testing with SQ4? I'm looking specifically for if this memory usage improvement translates with large number of vectors within a single index - say 100000+.

metonymic-smokey commented 4 months ago

@metonymic-smokey did you have a document of your testing with SQ4? I'm looking specifically for if this memory usage improvement translates with large number of vectors within a single index - say 100000+.

@abhinavdangeti posted this on the linked ticket.

thanks!

abhinavdangeti commented 4 months ago

Thanks @metonymic-smokey 👍🏼

abhinavdangeti commented 4 months ago

Rebased, and bring in https://github.com/blevesearch/bleve_index_api/pull/49 .