Warmers do not load field data cache for sorting on new segments

[bleskes]

Hi,

One the use cases for warmers is to pre-load the fields data cache with fields used for sorting.

The documentation doesn't specify how to load a field for sorting, but the following loads the id field into the cache if you put a new warmer after data has already been indexed:

PUT to /tests/_warmer/warmer1

{
    "query": {
        "match_all": {}
    },
    "sort": [
        { "id": { "order" : "desc" }}
    ]

}

However, new segments will not have the id field loaded.

To reproduce follow this sequence (on a fresh node). I tried it on 0.20.4: Create an empty tests index with the folldwing mapping:

{
   "test" : {
       "properties" : {
           "id" : { "type": "string" }
       }
   }
}

• Load the warmer: PUT to /tests/_warmer/warmer1

{
    "query": {
        "match_all": {}
    },
    "sort": [
        { "id": { "order" : "desc" }}
    ]

}

• Load two documents with the following structure (different ids): PUT to /tests/test/2

{
    "id": "2"
}

• Call refresh: GET to _refresh
• Get the nodes stats. Cache is empty: GET to /_cluster/nodes/stats

            "cache": {
               "field_evictions": 0,
               "field_size": "0b",
               "field_size_in_bytes": 0,
               "filter_count": 0,
               "filter_evictions": 0,
               "filter_size": "0b",
               "filter_size_in_bytes": 0,
               "bloom_size": "0b",
               "bloom_size_in_bytes": 0,
               "id_cache_size": "0b",
               "id_cache_size_in_bytes": 0
            },

• Put the warmer in again: PUT to /tests/_warmer/warmer1

{
    "query": {
        "match_all": {}
    },
    "sort": [
        { "id": { "order" : "desc" }}
    ]

}

• Now the cache will be loaded: GET to /_cluster/nodes/stats :

"cache": {
               "field_evictions": 0,
               "field_size": "132b",
               "field_size_in_bytes": 132,
               "filter_count": 0,
               "filter_evictions": 0,
               "filter_size": "0b",
               "filter_size_in_bytes": 0,
               "bloom_size": "0b",
               "bloom_size_in_bytes": 0,
               "id_cache_size": "0b",
               "id_cache_size_in_bytes": 0
            },

Cheers, Boaz

NOTE: This may just be a documentation issue as I'm guessing how you should "warm" a sort field. If this is the wrong way, I'd be happy know.

[kimchy]

Yea, we use COUNT search type, which does not kick in sorting, will change it to regular search. Btw, its much simpler to just have regular curl recreation, so we can more easily run it.