meilisearch / milli

Search engine library for Meilisearch ⚡️
MIT License
464 stars 81 forks source link

Use an unstable algorithm for `grenad::Sorter` when possible #635

Closed loiclec closed 2 years ago

loiclec commented 2 years ago

Pull Request

What does this PR do?

Use an unstable algorithm to sort the internal vector used by grenad::Sorter whenever possible to speed up indexing.

In practice, every time the merge function creates a RoaringBitmap, we use an unstable sort. For every other merge function, such as keep_first, keep_last, etc., a stable sort is used.

loiclec commented 2 years ago

Result of the benchmarks:

group                                                                     indexing_grenad-sorter-unstable_25d66dc6    indexing_main_f8697075
-----                                                                     ----------------------------------------    ----------------------
indexing/-geo-delete-facetedNumber-facetedGeo-searchable-                 1.00    41.7±11.50ms        ? ?/sec         1.03     42.8±7.88ms        ? ?/sec
indexing/-movies-delete-facetedString-facetedNumber-searchable-           1.06      9.1±1.56ms        ? ?/sec         1.00      8.6±2.26ms        ? ?/sec
indexing/-movies-delete-facetedString-facetedNumber-searchable-nested-    1.69    23.7±31.95ms        ? ?/sec         1.00     14.1±3.11ms        ? ?/sec
indexing/-songs-delete-facetedString-facetedNumber-searchable-            1.00     42.1±5.07ms        ? ?/sec         1.08     45.6±7.98ms        ? ?/sec
indexing/-wiki-delete-searchable-                                         1.01   263.7±13.44ms        ? ?/sec         1.00   260.8±18.31ms        ? ?/sec
indexing/Indexing geo_point                                               1.00      47.4±0.62s        ? ?/sec         1.01      47.8±0.40s        ? ?/sec
indexing/Indexing movies in three batches                                 1.00      12.9±0.17s        ? ?/sec         1.03      13.2±0.08s        ? ?/sec
indexing/Indexing movies with default settings                            1.00      10.3±0.29s        ? ?/sec         1.06      10.9±0.26s        ? ?/sec
indexing/Indexing nested movies with default settings                     1.00       7.8±0.08s        ? ?/sec         1.04       8.0±0.36s        ? ?/sec
indexing/Indexing nested movies without any facets                        1.00       7.3±0.17s        ? ?/sec         1.02       7.5±0.25s        ? ?/sec
indexing/Indexing songs in three batches with default settings            1.00      48.4±0.81s        ? ?/sec         1.02      49.5±0.97s        ? ?/sec
indexing/Indexing songs with default settings                             1.00      44.1±1.36s        ? ?/sec         1.04      46.0±1.26s        ? ?/sec
indexing/Indexing songs without any facets                                1.00      41.9±0.76s        ? ?/sec         1.03      43.1±0.84s        ? ?/sec
indexing/Indexing songs without faceted numbers                           1.00      43.8±0.89s        ? ?/sec         1.02      44.6±0.61s        ? ?/sec
indexing/Indexing wiki                                                    1.00    869.9±20.14s        ? ?/sec         1.01    877.5±20.43s        ? ?/sec
indexing/Indexing wiki in three batches                                   1.00     935.7±5.74s        ? ?/sec         1.01     942.8±8.63s        ? ?/sec
indexing/Reindexing geo_point                                             1.01      15.8±0.26s        ? ?/sec         1.00      15.7±0.10s        ? ?/sec
indexing/Reindexing movies with default settings                          1.02   261.4±21.94ms        ? ?/sec         1.00   257.0±14.29ms        ? ?/sec
indexing/Reindexing songs with default settings                           1.00       4.1±0.06s        ? ?/sec         1.03       4.3±0.11s        ? ?/sec
indexing/Reindexing wiki                                                  1.00   1495.6±30.18s        ? ?/sec         1.00   1499.5±26.89s        ? ?/sec

As usual, the results looked a bit better on my computer. But it still pretty nice, we get a 6% speed up on the default movies bench :)

bors[bot] commented 2 years ago

Build succeeded: