search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.73k stars 1.8k forks source link

[BUG] AWS Neptune to OpenSearch Lambda Poller fails with exception: 'Action/metadata line [1] contains an unknown parameter [_type]' #7280

Open mcuervoe opened 1 year ago

mcuervoe commented 1 year ago

Describe the bug The lambda created to ingest Neptune data into OpenSearch fails with exception ''Action/metadata line [1] contains an unknown parameter [_type]' when the OpenSearch engine was migrated from ElasticSearch 7.9 to OpenSearch 2.5

To Reproduce Steps to reproduce the behavior:

  1. Create an AWS Neptune Instance (version 1.2)
  2. Create an OpenSearch Instance (version 2.5)
  3. Install Amazon Neptune-to-OpenSearch replication setup (python3.9)
  4. Insert data into AWS Neptune
  5. Wait for Step function to kick-off and see error in CloudWatch and/or Step Functions

Expected behavior No error should happen. Data from Neptune should be indexed in OpenSearch

Plugins N/A

Screenshots If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

Additional context version ElasticSearch 7.9 worked without issues. The problem appeared when we migrated to OpenSearch 2.5

dblock commented 1 year ago

I think this is because mapping type parameters were removed in 2.x, see https://opensearch.org/docs/2.0/breaking-changes/.

Are you following https://docs.aws.amazon.com/neptune/latest/userguide/full-text-search-cfn-create.html? I imagine we need to go fix it to support OpenSearch 2.x. I opened https://github.com/awslabs/amazon-neptune-tools/issues/316.