openshift-cs / managed-openshift

Public roadmaps for the Red Hat Managed OpenShift offerings OpenShift Dedicated (OSD) and Red Hat OpenShift Service on AWS (ROSA)
Apache License 2.0
56 stars 6 forks source link

Support for log forward to Amazon OpenSearch using Sigv4 and/or Basic Auth #133

Open omni1504 opened 1 year ago

omni1504 commented 1 year ago

Which service is this feature request for? Both OpenShift Dedicated and Red Hat OpenShift Service on AWS (preference on the latter)

What are you trying to do? Currently OCP/ROSA does not support AWS Opensearch as a Log Output type (https://access.redhat.com/solutions/6963894), however it does support Elasticsearch. Basically, there is no way to natively forward container logs from one native AWS service to another. Many Customers would use a managed AWS Opensearch/Opensearch Serverless as their log aggregator, making it a natural case/destination for ROSA logs.

Describe the solution you'd like Option 1 (probably much faster to implement): Implement Basic Auth scheme (username/password , non-Sigv4 signed) authentication for a native Opensearch Log output. Most likely will be pretty much the same as for currently supported Elasticsearch.

Option 2: implement Sigv4 support (similar to Opensearch plugins for Fluentbit/Logstash/etc).

Describe alternatives you've considered Sending logs first to Logstash, then, via Logstash's Opensearch plugin, to AWS Opensearch - that adds one more moving part to the equation, impacting availability and operability.

maulik-modi22 commented 4 days ago

@omni1504, Have you tried using OTLP Exporter in Openshift-Logging 6.0 - https://docs.openshift.com/container-platform/4.17/observability/logging/logging-6.0/log6x-release-notes.html#log6x-release-notes-6-0-0-technology-preview-features