filodb / FiloDB

Distributed Prometheus time series database
Apache License 2.0
1.43k stars 225 forks source link

feat(core): Add support for Tantivy based time series index #1852

Closed rfairfax closed 4 weeks ago

rfairfax commented 1 month ago

Pull Request checklist

New behavior :

This change adds support for the Tantivy indexing library as an alternative to Lucene for time series indexing. In several cases it has been found that this is superior to Lucene performance, especially when it comes to memory usage and predictability of memory spikes.

This feature is opt-in via a configuration setting to avoid any unexpected changes during upgrade. For the moment only the raw time series index is supported. Downsample support may come in a future PR.

BREAKING CHANGES

This change requires a working Rust & C compiler to build given the Tantivy code is written in Rust. README docs have been updated to reflect this.

There are no runtime breaking changes.