New component: eventhubsexporter

JakeDern commented 1 year ago

The purpose and use-cases of the new component

I am currently developing an Azure Event Hubs exporter which is reasonably far along. If there is interest we would love to contribute it back to this project.

Some of the motivations for doing so:

Take advantage of more authentication methods available in the azure sdk for go project
Have clear configuration that is designed for event hubs/azure rather than being molded around Kafka configuration
Avoid risk of issues like #11846 where Kafka client libraries like Sarama are not necessarily considering event hubs in their development

Some additional implementation details/goals:

Based on azeventhubs library in the azure sdk
Leverages the exporterhelper for common functionality
Supports common encodings like otlp_proto and otlp_json, but allows for easily extending to include additional encodings in custom distributions
Supports some common compression options for the payload, but also allows for easily extending to include additional options/implementations in custom distributions

Example configuration for the component

Here's a sample of the current configuration options. For auth we've just started with a connection string with shared access key, but plan to expand on that soon:

// Auth defines authentication options for the event hubs exporter.
type Auth struct {
    ConnectionString string `mapstructure:"connection_string"`
}

// Config defines configuration for OTLP exporter.
type Config struct {
    // Auth defines the authentication options for the event hubs exporter
    Auth Auth `mapstructure:"auth"`

    // Encoding for the payload e.g. "otlp_json" or "otlp_proto"
    Encoding string `mapstructure:"encoding"`

    // Compression for the payload e.g. "snappy"
    Compression string `mapstructure:"compression"`

    // Inherited settings from exporterhelper
    exporterhelper.TimeoutSettings `mapstructure:",squash"`
    exporterhelper.RetrySettings   `mapstructure:"retry_on_failure"`
    exporterhelper.QueueSettings   `mapstructure:"sending_queue"`
}

Telemetry data types supported

Logs, metrics, and traces are all supported.

Is this a vendor-specific component?

[X] This is a vendor-specific component
[x] If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

No response

Additional context

While I work for Microsoft within Azure, I do not work on the azeventhubs client library nor on Event Hubs itself. Rather I am developing a different product that makes use of both OpenTelemetry and Event Hubs.

MovieStoreGuy commented 1 year ago

I am happy to sponsor this provided we get another contributor from Azure to be code owners.

JakeDern commented 1 year ago

@MovieStoreGuy Do you need someone involved as a contributor in the project already, or do you just need a couple of names from us to put down in the codeowners file?

If you need someone who is already a contributor on the project I can try to reach out to some folks listed on the other azure components in here.

MovieStoreGuy commented 1 year ago

@MovieStoreGuy Do you need someone involved as a contributor in the project already

They need to be community members for them to be listed as a code owner. I am happy to provide the technical direction and support but I can't speak on behalf of Azure so I would like more people to help answer those types of questions.

JakeDern commented 1 year ago

@MovieStoreGuy I looked through the other azure-related components in the repo and the owners listed there who are still at Microsoft don't seem to be in the open-telemetry organization. If I can get in touch with them would that work or should I be looking for members only?

Also would the slack be an appropriate forum to advertise that we're looking for owners for this component?

atoulme commented 1 year ago

Slack is a great place to start. Feel free to come to a SIG meeting as well. Some context on codeowners: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/20868 Some context on Azure-related codeowners:

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

github-actions[bot] commented 12 months ago

This issue has been closed as inactive because it has been stale for 120 days with no activity.

open-telemetry / opentelemetry-collector-contrib