jprante / elasticsearch-langdetect

A plugin for language detection in Elasticsearch using Nakatani Shuyo's language detector
Apache License 2.0
251 stars 46 forks source link

mapper_parsing_exception for some languages #79

Closed mdahamiwal closed 6 years ago

mdahamiwal commented 6 years ago

Facing issue while pushing document for few languages like french, spanish and chinese. I haven't tried for all languages though. The language detection endpoint returns expected language but it throws while pushing the same text to the index.

ES version: 5.4.1

Mapping:

PUT /test
{
   "mappings": {
      "docs": {
         "properties": {
            "content": {
               "type": "langdetect",
               "languages": [
                  "fr",
                  "en",
                  "es"
               ]
            }
         }
      }
   }
}

Add document:

Put test/doc/1
{
  "content": "Maître Corbeau, sur un arbre perché"
}

Output:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "analyzer [_keyword] not found for field [content]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "analyzer [_keyword] not found for field [content]"
  },
  "status": 400
}

=================================================================== Language detection:

{
   "text":  "Maître Corbeau, sur un arbre perché"
}

Output:

  "languages": [
    {
      "language": "fr",
      "probability": 0.9999965912615383
    }
  ]
}