opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.74k stars 1.81k forks source link

[BUG] ignore_malformed doesn't work for scaled_float #2897

Closed tszajna0 closed 2 years ago

tszajna0 commented 2 years ago

Describe the bug When index has mapped field of type scaled_float with ignore_malformed set, and a document is inserted with value of this field equal to false, then mapper_parsing_exception is thrown.

Interestingly, this bug isn't reproduced for a regular float fields.

To Reproduce Steps to reproduce the behavior:

  1. Create an index

    PUT {{ENDPOINT}}/test { "mappings": { "properties": { "amount": { "type": "scaled_float", "ignore_malformed": true, "scaling_factor": 100.0 } } }, "settings": { "index.mapping.ignore_malformed": "true" } }

  2. Try to insert document

    POST {{ENDPOINT}}/test/_doc { "amount": false }

  3. See error

    { "error": { "root_cause": [ { "type": "mapper_parsing_exception", "reason": "failed to parse field [amount] of type [scaledfloat] in document with id 'iL1EJ4ABZDKYwbEvExL'. Preview of field's value: 'false'" } ], "type": "mapper_parsing_exception", "reason": "failed to parse field [amount] of type [scaledfloat] in document with id 'iL1EJ4ABZDKYwbEvExL'. Preview of field's value: 'false'", "caused_by": { "type": "json_parse_exception", "reason": "Current token (VALUE_FALSE) not numeric, can not use numeric value accessors\n at [Source: (byte[])\"{\n \"amount\": false\n}\"; line: 2, column: 20]" } }, "status": 400 }

Expected behavior

Plugins Plugins installed by default by AWS OpenSearch:

   name                             component                            version
   1e7975406315f4b7e4cce0fec3113412 analysis-icu                         1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-ik                          1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-kuromoji                    1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-phonetic                    1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-seunjeon                    1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-smartcn                     1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-stempel                     1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-thaichub2                   1.2.4
   1e7975406315f4b7e4cce0fec3113412 analysis-ukrainian                   1.2.4
   1e7975406315f4b7e4cce0fec3113412 discovery-ec2                        1.2.4
   1e7975406315f4b7e4cce0fec3113412 elasticsearch-aes-iam                1.0.0
   1e7975406315f4b7e4cce0fec3113412 elasticsearch-aes-remote-reindex     1.2.4
   1e7975406315f4b7e4cce0fec3113412 ingest-attachment                    1.2.4
   1e7975406315f4b7e4cce0fec3113412 kraken                               unspecified
   1e7975406315f4b7e4cce0fec3113412 ltr                                  1.5.5-opensearch-1.2.4
   1e7975406315f4b7e4cce0fec3113412 mapper-murmur3                       1.2.4
   1e7975406315f4b7e4cce0fec3113412 mapper-size                          1.2.4
   1e7975406315f4b7e4cce0fec3113412 opensearch-alerting                  x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch-analysis-vietnamese       1.2.4
   1e7975406315f4b7e4cce0fec3113412 opensearch-anomaly-detection         x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch-asynchronous-search       x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch-cross-cluster-replication x.x.x.x
   1e7975406315f4b7e4cce0fec3113412 opensearch-index-management          x.x.x.x
   1e7975406315f4b7e4cce0fec3113412 opensearch-jetty                     2.2.0
   1e7975406315f4b7e4cce0fec3113412 opensearch-job-scheduler             x.x.x.x
   1e7975406315f4b7e4cce0fec3113412 opensearch-knn                       x.x.x.x
   1e7975406315f4b7e4cce0fec3113412 opensearch-observability             x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch-reports-scheduler         x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch-security                  x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch-sql                       x.x.x.x-SNAPSHOT
   1e7975406315f4b7e4cce0fec3113412 opensearch_cross_cluster             1.2.4
   1e7975406315f4b7e4cce0fec3113412 performance-analyzer                 x.x.x.x
   1e7975406315f4b7e4cce0fec3113412 repository-s3                        1.2.4
   1e7975406315f4b7e4cce0fec3113412 sifi-remediation-plugin              x.x.x.x

Screenshots not applicable

Host/Environment (please complete the following information):

Additional context not applicable

saratvemulapalli commented 2 years ago

@tszajna0 thanks for reporting this. I was able to reproduce the problem with instructions. The implementation of float and scaled_float is different causing this edge case to miss. I've sent out #2918 to fix it.