Update @searchable documentation to recommend explicit ElasticSearch Mapping

[sacrampton]

Is your feature request related to a problem? Please describe.

This is related to aws-amplify/amplify-cli#4436 where the current Lambda sync function from DDB to ES infers data types based on data sent to ES rather than honouring the graphql schema. For example if you have userField01: String and you send a string with the value of "45.71" to ES in the first record it will create userField01: Float in ES - so when you go to send "Test" in the second record it will fail.

I was having records failing so dug into this problem and decided that I needed to create explicit ElasticSearch mappings that aligned with my schema.graphql. I have 44 tables in my database with thousands of fields and have separate dev, test, prod environments. Took a very long time to generate and apply all these ElasticSearch mappings, but the result shocked me. Approximately half of my tables failed to update with the mappings derived from schema.graphql - so I had to re-index those indexes with the correct mappings (see below).

PUT /asset/_mapping/doc
{
  "properties": {
    "id": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    },
    "acquisitionDate": {
      "type": "date"
    },
    "acquisitionValue": {
      "type": "float"
    },
    "assetVisitQueue": {
      "type": "integer"
    },
    "auditCompleted": {
      "type": "boolean"
    },
    "geolocation": {
      "type": "geo_point"
    }
  }
}

Describe the solution you'd like Ideally given the significant vulnerability this introduces for inconsistency between DDB and ES I think Amplify/AppSync should generate an explicit mapping template that aligns with schema.graphql and apply it so that the solution is not open to random data types being inferred.

However, in the absence of this I think there should be very clear warnings in the documentation that there is a high risk of incompatible data types being created between DDB & ES and that if you are using @searchable you MUST manually apply explicit mappings to ElasticSearch.

Describe alternatives you've considered I solved this using explicit mappings in ES - but is was a MASSIVE amount of work.

[attilah]

@sacrampton thanks for reporting this it seems to be a possible enhancement, documentation and better DX issue. Two of them is easy and simple, the mapping generation is something that requires a bigger effort we'll keep track of this and glad that you put in the hard work and figured it out we appreciate it.

[brauliolledo]

+1 to this. We've come across this issue affecting a handful of production environments for our Amplify app and causing some tables to not be indexed and therefore records being missing from queries.