Closed davidshtian closed 1 year ago
elfisher
commented
3 years ago Hey @davidshtian thanks for sharing this request. For now, the alternative is to use the Elasticsearch Flink connector that is compatible with v7.10.
With that said, adding a dedicated OpenSearch connector would be a welcome contribution. I'll tag this as help wanted and see if someone could help. If you'd be interested in helping, let us know.
reta
commented
2 years ago @elfisher @davidshtian we have started the discussion to introduce the dedicated connector [1], [2] (is related to that), @dblock if there are no objections, please assign this issue to me, thank you.
[1] https://issues.apache.org/jira/browse/FLINK-25756 [2] https://github.com/testcontainers/testcontainers-java/issues/4782
ypark2103
commented
1 year ago Hi @reta Any updates on when the dedicated OpenSearch connector for flink will be released?
reta
commented
1 year ago @ypark2103 https://github.com/apache/flink/pull/18541
ypark2103
commented
1 year ago @reta So can you please provide ETA when we will be able to see OpenSearch connector in official flink document and merged to master?
reta
commented
1 year ago @reta So can you please provide ETA when we will be able to see OpenSearch connector in official flink document and merged to master?
@ypark2103 I wish I could, I have no control over it, the pull request in question is awaiting review from Apache Flink team
norrishuang
commented
1 year ago Did flink connector support aws opensearch serverless?
reta
commented
1 year ago @norrishuang not out of the box but should be feasible using [1] and RestClientFactory (depending what sink you are using, you may need to wait until [2])
[1] https://github.com/acm19/aws-request-signing-apache-interceptor [2] https://issues.apache.org/jira/browse/FLINK-31856
reta
commented
1 year ago Closing this issue, the connector is added: https://github.com/apache/flink-connector-opensearch
Is your feature request related to a problem? Please describe. As OpenSearch is the fork of Elasticsearch, is there any plan to support Flink OpenSearch Connector in the future? Like the ones already build for Elasticsearch (Elasticsearch Connector and Elasticsearch SQL Connector).
Describe the solution you'd like Add a new version of Flink OpenSearch connector to keep updated with OpenSearch development.
Describe alternatives you've considered Use Flink Elasticsearch Connector as workaround, but not sure the compatibility in later versions.
Additional context This feature request is for the ecosystem of OpenSearch, as the OpenSearch is not a standalone component and it definitely needs upstream and downstream software for the total solution.
Thanks~