elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.95k stars 24.74k forks source link

Request cache is not populated when using warmer API #15194

Closed spinscale closed 8 years ago

spinscale commented 8 years ago

It looks as the request cache is not populated when registering a warmer and the index has configured index.requests.cache.enable. Reproduce like this

PUT /test
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0,
    "index.requests.cache.enable": true
  },
  "warmers": {
    "warmer": {
      "source": {
        "aggs": {
          "aggs_1": {
            "terms": {
              "field": "field"
            }
          }
        }
      }
    }
  }
}

# Check stats, hit count should be 0
GET /test/_stats/request_cache?filter_path=indices.test.total.request_cache

# refresh runs the warmer
PUT /test/test/1?refresh
{
  "field" : "somevalue"
}

# Check stats, hit count should be 0 again
GET /test/_stats/request_cache?filter_path=indices.test.total.request_cache

# Execute agg
GET /test/_search?search_type=count
{
  "aggs": {
    "foo": {
      "terms": {
        "field": "field",
        "size": 10
      }
    }
  }
}

# PROBLEM HERE: Check stats, hit count should be 1, but is 0, missing count is increased
GET /test/_stats/request_cache?filter_path=indices.test.total.request_cache

# Execute agg a second time
GET /test/_search?search_type=count
{
  "aggs": {
    "foo": {
      "terms": {
        "field": "field",
        "size": 10
      }
    }
  }
}

# Finally we have a cache hit but too late
GET /test/_stats/request_cache?filter_path=indices.test.total.request_cache

The issue seems particularly troubling to me, because this means, that the user will always get one slow query (and I do not see any external possibility to prevent this, like running the query manually and hoping it gets executed before the first user query hits). This means there will always be a slow agg, if I understood it correctly.

This also happens with the query shard cache in 1.7.

jpountz commented 8 years ago

I would agree it's a shame that it does not get cached. But we have several ideas to remove warmers, and in-particular query-based warmers (#9331) so I wouldn't spend too much energy on fixing it.

jpountz commented 8 years ago

Closing: warmers will go away in 3.0 #15614