search

quickwit-oss / tantivy

Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust
MIT License
12.03k stars 671 forks source link

Implement date field type. #42

Closed fulmicoton closed 5 years ago

manuel-woelker commented 7 years ago

What would implementation options be?

This type probably needs to support range queries, e.g. "only return events since midnight"

Possibly Relevant link: https://www.elastic.co/blog/apache-lucene-numeric-filters

It seems Lucene uses block KD trees, which might also enable LatLon types.

fulmicoton commented 7 years ago

@manuel-woelker sorry for the late answer.

We definitely want to use a 64 bit representation for that. Maybe i64 rather than u64 however? We probably want to implement i64 or u64 and i64 fields and just wrap dates over such a field. Then we should probably ditch the u32. Having a specific u32 field should not save any memory as tantivy use bitpacking in fastfields. (Implementing the reader in an efficient way might be a bit more challenging though)

Then comes another question which is how to make ranges searchable. I think this should a different ticket.

I'll create another one.

Tantivy needs to support range for other numerical types as well, so the capacity to index a development over the integer. I am unsure what we want here.

fulmicoton commented 7 years ago

@manuel-woelker let me know if you want to work on these ticket.

barrotsteindev commented 5 years ago

We probably want to implement i64 or u64 and i64 fields and just wrap dates over such a field.

Would we have a query parser parse dates to timestamps, or is that up to the user to parse string dates?