Bit-Quill / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
0 stars 0 forks source link

New component: OpenSearch Exporter [WIP] #16

Open MitchellGale opened 1 year ago

MitchellGale commented 1 year ago

The purpose and use-cases of the new component

OpenSearch is used for real-time application monitoring, log analytics, website search and more. The OpenSearch exporter would convert OpenTelemetry data directly into OpenSearch allowing logs, trace, metrics to be ingested and searched using OpenSearch. Exporting OpenTelemetry data into OpenSearch will help improve observability.

Many users are currently using OpenSearch as store repository for logs and need a common way and a standard protocol to consolidate both logs and trace signals.

Example configuration for the component

exporters:
  opensearch/trace:
    keyspace: "otel"
    endpoints: [https://localhost:9200]
    traces_index: trace_index
  opensearch/log:
    endpoints: [http://localhost:9200]
    logs_index: my_log_index
    sending_queue:
      enabled: true
      num_consumers: 20
      queue_size: 1000
······
service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [opensearch/log]
    traces:
      receivers: [otlp]
      exporters: [opensearch/trace]
      processors: [batch]

Telemetry data types supported

Is this a vendor-specific component?

Sponsor (optional)

Anthony Mirabella (Aneurysm9)

Additional context

Will start contributing with traces initially then follow up with logs.

OpenSearch has recently forked and showed the usage of OpenSearch as a backend store and exploration tools on top of the open telemetry astronomer shop demo using simple schema as the mapping physical store of the OTEL signals

https://github.com/opensearch-project/opentelemetry-demo https://github.com/opensearch-project/opensearch-catalog/tree/main/schema/observability https://github.com/opensearch-project/opensearch-catalog/tree/main/docs/schema

I will be directly contributing to this. I am open to becoming a member of OpenTelemetry as progress on this exporter progresses.

YANG-DB commented 1 year ago

OpenSearch has recently forked and showed the usage of OpenSearch as a backend store and exploration tools on top of the open telemetry astronomer shop demo using simple schema as the mapping physical store of the OTEL signals

additional context: