replikativ / datahike

A fast, immutable, distributed & compositional Datalog engine for everyone.
https://datahike.io
Eclipse Public License 1.0
1.62k stars 95 forks source link

Optimize the persistent set comparator #672

Closed jonasseglare closed 3 months ago

jonasseglare commented 3 months ago

This PR rewrites the slice comparator for the persistent set index backend to be much faster. It accomplishes that by doing most of the work at macro expansion time to generate an inlined implementation of the comparator. Here are the absolute and relative times measured using this benchmark.

TARGET              ABS TIME (s)   REL TIME
Some other db              4.797         6%
Official Datahike         75.648       100%
Datahike pset cmp         53.027        70%

This PR is the line "Datahike pset cmp". We see that it takes about 30% less time to run the benchmark using the code in this PR compared to the code in the main branch.