analyzers on non-string fields do not migrate correctly to 2.2

[clintongormley]

From @plebedev on April 1, 2016 19:49

Elasticsearch version: 2.2.0

JVM version: 1.7 or 1.8

OS version: OSX 10.9.5 but it does not really matter, issue is reproducible on various Linux flavors as well.

Description of the problem including expected versus actual behavior: Looks like there was a change in 2.2 related to analyzers. In 2.0 you can specify analyzer and search_analyzer on any filed type. In 2.2 you can only do it for 'string' types. It maybe makes sense but it is not compatible with previous versions and many cases fail in different way.

Steps to reproduce: With 2.0.2, create a new index:

PUT index_try123
{
  "mappings": {
    "doc": {
      "properties": {
        "prop1": {
          "type": "boolean",
          "analyzer": "standard"
        },
        "prop2": {
          "type": "string",
          "analyzer": "standard",
          "search_analyzer": "standard"
        }
      }
    }
  }
}

Upgrade ES to 2.2.1

Try to update the mapping:

PUT index_try123/_mapping/doc
{
  "properties": {
        "prop1": {
          "type": "boolean"
        },
        "prop2": {
          "type": "string",
          "analyzer": "standard"
        },
        "prop3": {
          "type": "string",
          "analyzer": "standard"
        }
      }
}

This fails with the error:

{
   "error": {
      "root_cause": [
         {
            "type": "mapper_parsing_exception",
            "reason": "analyzer [_keyword] not found for field [prop1]"
         }
      ],
      "type": "mapper_parsing_exception",
      "reason": "analyzer [_keyword] not found for field [prop1]"
   },
   "status": 400
}

If you look at the index after you created it with 2.0.1, you will see "search_analyzer": "_keyword" added to prop1.

Now, delete the index and try to create it again:

DELETE index_try123

PUT index_try123
{
  "mappings": {
    "doc": {
      "properties": {
        "prop1": {
          "type": "boolean",
          "analyzer": "standard"
        },
        "prop2": {
          "type": "string",
          "analyzer": "standard",
          "search_analyzer": "standard"
        }
      }
    }
  }
}

This now fails with this error:

{
   "error": {
      "root_cause": [
         {
            "type": "mapper_parsing_exception",
            "reason": "Mapping definition for [prop1] has unsupported parameters:  [analyzer : standard]"
         }
      ],
      "type": "mapper_parsing_exception",
      "reason": "Failed to parse mapping [doc]: Mapping definition for [prop1] has unsupported parameters:  [analyzer : standard]",
      "caused_by": {
         "type": "mapper_parsing_exception",
         "reason": "Mapping definition for [prop1] has unsupported parameters:  [analyzer : standard]"
      }
   },
   "status": 400
}

So, if you had a mapping with an analyzer set on non-string field in 2.0, and then migrate to 2.2, you can't update existing mappings, nor you can create the same mapping on a fresh installation without modifying the put request.

Copied from original issue: elastic/elasticsearch#17485

[clintongormley]

Hi @plebedev

Your issue is correct, but is a real edge case. I think it's one for the migration tool rather than anything we plan to fix in elasticsearch. I'll move it there.

[clintongormley]

ES 2.3 rewrites these mappings to a valid form, so I'm going to close this