quickwit-oss / tantivy

Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust
MIT License
11.8k stars 655 forks source link

SnippetGenerator doesn’t work with FuzzyTermQuery #867

Closed b8591340 closed 4 years ago

b8591340 commented 4 years ago

Describe the bug SnippetGenerator doesn’t work with FuzzyTermQuery

Which version of tantivy are you using? 0.13.0

To Reproduce

use tantivy::collector::*;
use tantivy::query::*;
use tantivy::schema::*;
use tantivy::*;

fn main() {
    let mut schema_builder = Schema::builder();
    let text_field = schema_builder.add_text_field("text", TEXT);
    let schema = schema_builder.build();
    let index = Index::create_in_ram(schema);
    let mut index_writer = index.writer_with_num_threads(1, 30_000_000).unwrap();
    let doc = doc!(text_field => r#"Comme je descendais des Fleuves impassibles,
           Je ne me sentis plus guidé par les haleurs :
          Des Peaux-Rouges criards les avaient pris pour cibles,
          Les ayant cloués nus aux poteaux de couleurs.

        J'étais insoucieux de tous les équipages,
          Porteur de blés flamands ou de cotons anglais.
          Quand avec mes haleurs ont fini ces tapages,
          Les Fleuves m'ont laissé descendre où je voulais.
          "#);
    index_writer.add_document(doc.clone());
    index_writer.commit().unwrap();
    let query = Box::new(FuzzyTermQuery::new_prefix(
        Term::from_field_text(text_field, "haleu"),
        2,
        true,
    )) as Box<dyn Query>;
    let reader = index.reader().unwrap();
    let searcher = reader.searcher();
    let count = searcher.search(&query, &Count).unwrap();
    assert_eq!(count, 1);
    let mut snippet_generator = SnippetGenerator::create(&searcher, &query, text_field).unwrap();
    snippet_generator.set_max_num_chars(100);
    let text: String = doc
        .get_all(text_field)
        .into_iter()
        .flat_map(Value::text)
        .collect::<Vec<&str>>()
        .join(" ");
    let snippet = snippet_generator.snippet(&text);
    assert!(snippet.highlighted().is_empty());
}
b8591340 commented 4 years ago

Duplicate #672