Custom DdbToEsFn ElasticSearch indexer lambda implementation

[PatrykMilewski]

Is your feature request related to a problem? Please describe. We are using Amplify framework with our custom DdbToEsFn function implementation. Due to this we were forced to give up on @searchable annotation and implement custom resolvers for requests, that needs indexes.

Describe the solution you'd like I would like to be able to point to my custom indexing function via some sort of configuration. The ideal way would be to simply provide ARN of function, that Amplify should use.

Describe alternatives you've considered The alternative is our current implementation, which is not really comfortable and flexible. Maybe it's possible to achieve this using Plugins: https://docs.amplify.aws/cli/plugins But I'm not really an expert about that.

[ammarkarachi]

Hey @PatrykMilewski, sorry to hear that you had difficulty using your own function. You can modify the function under amplify/backend/api/myAPI/build/functions/ElasticSearchStreamingLambdaFunction.zip

[ammarkarachi]

@PatrykMilewski As a way to collect more feedback for our @searchable directive, what gaps are you trying to fill that current solution isn't?

[PatrykMilewski]

Hey @PatrykMilewski, sorry to hear that you had difficulty using your own function. You can modify the function under amplify/backend/api/myAPI/build/functions/ElasticSearchStreamingLambdaFunction.zip

Oh I didn't know that! I will test it and check if it will be working for me, thanks!

We wanted to filter results from Elasticsearch query based on nested field, that was present in one of models, example:

type Movie {
  id: String!
  availability: [Availability]
}

type Availability {
  start: String
  name: String!
}

With default implementation we were not able to search for movies, that has availability of given name. Pretty much that's the limitation of filtering by nested fields.

Also splitting some strings by words into array to have better autocompletion in search + search endpoints with fuzziness for typos.

[MontoyaAndres]

Hey @PatrykMilewski, sorry to hear that you had difficulty using your own function. You can modify the function under amplify/backend/api/myAPI/build/functions/ElasticSearchStreamingLambdaFunction.zip

Hello, How can I modify it? I just need to create the function in amplify/backend/api/myAPI/functions/MyNewFunction.js?

[aleksey-shmatov]

Just putting file ElasticSearchStreamingLambdaFunction.zip with modified python function under amplify/backend/api/myAPI/functions worked for me to update streaming function. However quite often any changes there would also might require changes to schema(custom query), and request/response resolvers. In my case I just wanted to avoid adding unnecessary data into elastic search index and just add a few top level fields. However that did not work well, index is still created with all the fields specified in the schema and I have to write custom resolver to fetch this omitted data from DynamoDb. In the end I have decided to split model in two - Item and Details. Item would contain minimal amount of data required for search and details could be pulled via connection. I am not very happy with this solution, now I have to use TransactWrite for updating both tables at the same time and have additional connection which in my case always have to be resolved. Better for me would be to have some sort of control and specify searchable related directives per fields(make some fields unsearchable).