search

awslabs / fhir-works-on-aws-persistence-ddb

A DynamoDB implementation of the FHIR Works on AWS framework, enabling users to complete CRUD operations on FHIR resources
Apache License 2.0
27 stars 22 forks source link

[Misc] elasticsearch-js compatibility with OpenSearch #118

Closed Zambonilli closed 3 years ago

Zambonilli commented 3 years ago

What's on your mind? Sorry, this might be a dumb question and not urgent. How is the current ddbToEsSync logic connecting to OpenSearch clusters without error when using elasticsearch-js 7.4? Elastic added a gem to their connection logic in elasticsearch-js to fail the connection if the cluster responds saying that the cluster is running a version that is licensed under oss. When using the same exact versions of @elastic/elasticsearch (^7.4), aws-elasticsearch-connector (^8.2.0) and aws-sdk (^2.610.0) in a fresh node app I receive the OSS error trying to connect to our fwoa elastic cluster. The recommended fix is to pin to @elastic/elasticsearch version 7.13 and it looks like the OpenSearch team is working on a fork that works with OSS. With my new node app, rolling back to 7.13 fixed the connect failure but I'm wondering as to how we're not seeing the same error in fhir-works-on-aws-persistence-ddb when it's using @elastic/elasticsearch. I didn't see any pushes mentioning the failure and the connect code doesn't look to have any special logic.

https://github.com/elastic/elasticsearch-js/issues/1519 https://aws.amazon.com/blogs/opensource/keeping-clients-of-opensearch-and-elasticsearch-compatible-with-open-source/

Versions (please complete the following information):

carvantes commented 3 years ago

Hi,

The official guidance is here: https://opensearch.org/docs/clients/index/. The latest client that works fine is 7.13.0

@elastic/elasticsearch ^7.4 is not ideal since ^7.4 is "greater or equal to 7.4.0 without bumping the major version", so a fresh install will get 7.15.0 and have issues with the client.

It hasn't been an apparent issue for FHIR works since it is pinned to an older version in our yarn.lock file: https://github.com/awslabs/fhir-works-on-aws-deployment/blob/4e0b5ed2df673c636293264ebbafff5df6ce9ee6/yarn.lock#L1071-L1072

I created a PR to pin the elasticsearch client version on this package to 7.13.0: https://github.com/awslabs/fhir-works-on-aws-persistence-ddb/pull/120

carvantes commented 3 years ago

We just released fhir-works-on-aws-persistence-ddb@3.8.2 with the above fix

Zambonilli commented 3 years ago

Awesome, thank you for your detailed response. I thought I was taking crazy pills staring at the version numbers too long. I completely forgot that the yarn lock pins the versions too.