mosuka / blast

Blast is a full text search and indexing server, written in Go, built on top of Bleve.
Apache License 2.0
1.08k stars 76 forks source link

How to search documents with id prefix? #137

Open lauthrul opened 4 years ago

lauthrul commented 4 years ago

I have two kind of documents, A and B, I index them in bulk with different document id prefix. basically like:

{"fields":{"code":"xxx","name":"xxx"},"id":"path/A/10"}
{"fields":{"code":"xxx","name":"xxx"},"id":"path/A/11"}
{"fields":{"code":"xxx","name":"xxx"},"id":"path/B/20"}
{"fields":{"code":"xxx","name":"xxx"},"id":"path/B/21"}

Now I only want to search documents with id prefix "path/A/". How to do this?

mosuka commented 4 years ago

Hi @lauthrul , id can only be used to search for exact matches. Why don't you add the following field to search there?

{"fields":{"code":"xxx","name":"xxx","path":"path/A/10"},"id":"path/A/10"}
{"fields":{"code":"xxx","name":"xxx","path":"path/A/11"},"id":"path/A/11"}
{"fields":{"code":"xxx","name":"xxx","path":"path/B/20"},"id":"path/B/20"}
{"fields":{"code":"xxx","name":"xxx","path":"path/B/21"},"id":"path/B/21"}

Add the following field definitions to your schema:

        "path": {
          "enabled": true,
          "dynamic": true,
          "fields": [
            {
              "type": "text",
              "analyzer": "keyword",
              "store": true,
              "index": true,
              "include_term_vectors": true,
              "include_in_all": true
            }
          ],
          "default_analyzer": "keyword"
        },

You could search for the following query:

{
  "query": {
    "prefix": "path/A/",
    "field": "path"
  },
  "size": 10,
  "from": 0,
  "fields": [
    "*"
  ],
  "sort": [
    "-_score"
  ]
}
lauthrul commented 4 years ago

Hi @lauthrul , id can only be used to search for exact matches. Why don't you add the following field to search there?

{"fields":{"code":"xxx","name":"xxx","path":"path/A/10"},"id":"path/A/10"}
{"fields":{"code":"xxx","name":"xxx","path":"path/A/11"},"id":"path/A/11"}
{"fields":{"code":"xxx","name":"xxx","path":"path/B/20"},"id":"path/B/20"}
{"fields":{"code":"xxx","name":"xxx","path":"path/B/21"},"id":"path/B/21"}

Add the following field definitions to your schema:

        "path": {
          "enabled": true,
          "dynamic": true,
          "fields": [
            {
              "type": "text",
              "analyzer": "keyword",
              "store": true,
              "index": true,
              "include_term_vectors": true,
              "include_in_all": true
            }
          ],
          "default_analyzer": "keyword"
        },

You could search for the following query:

{
  "query": {
    "prefix": "path/A/",
    "field": "path"
  },
  "size": 10,
  "from": 0,
  "fields": [
    "*"
  ],
  "sort": [
    "-_score"
  ]
}

Thanks for reply. Because the document is unchangable, so I can't add id filed to the document. I found a way to search with id prefix, just add "+_id:path/A/*" to the query. But now there is another question. How to implement logic operatation(and, or) in the query conditon? For example, I want to search A kind documents which contains "xxx" in name or some other fileds(let's say short_name). this is like:

{
  "search_request": {
    "query": {
      "query": "+_id:path/A/* name:*xxx* short_name:*xxx*"
    },
    "fields": [
      "*"
    ],
    "from": 0,
    "size": 100,
    "sort": [
      "-_score"
    ]
  }
}

This will search all documents of A kind, condition "name:xxx short_name:xxx" didn't take effect. If change to "+name:xxx +short_name:xxx", nothing searched. As bleve offical documents says, "+" means MUST, so this is mean name or short_name is a MUST or MUST NOT meet condition. However, what I want is OR and AND condition, that is, I want xxx in name OR short_name, AND with id prefix "path/A/". Do you have any ideas of how to implement this?

mosuka commented 4 years ago

@lauthrul Bleve's QueryStringQuery may not be able to express complex queries. You may need to use a combination of BooleanQuery or similar.

http://blevesearch.com/docs/Query-String-Query/