New component: Doris Exporter

joker-star-l commented 3 months ago

The purpose and use-cases of the new component

The Doris Exporter would allow for the export of logs, metrics, and traces to Apache Doris, a modern data warehouse for real-time analytics. The Doris Exporter aims to provide a seamless way to export observability data from OpenTelemetry-collected applications to Doris. This component will enable users to take advantage of Doris’s powerful analytics capabilities to gain insights into their application’s performance and behavior.

Example configuration for the component

Configuration options:

endpoint (no default): The Doris FE address.
database (default otel): The database name.
username (default ): The authentication username.
password (default ): The authentication password.
create_schema (default true): Whether databases and tables are created automatically.
logs_table_name (default otel_logs): The table name for logs.
traces_table_name (default otel_traces): The table name for traces.
metrics_table_name (default otel_metrics): The table name for metrics.
ttl (default 0): The data time-to-live, 0 means no ttl.
timeout (default 5s): The timeout for each data send.
retry
- enabled (default true): Whether retries are allowed.
- initial_interval (default 5s): Retry wait time after the first failure. (Ignored if enabled is false.)
- max_interval (default 30s): The upper bound on backoff. (Ignored if enabled is false.)
- max_elapsed_time (default 100s): The maximum time spent trying to send a batch. (Ignored if enabled is false.)

Example:

exporters:
  doris:
    endpoint: http://doris:8030
    database: otel
    username: username
    password: password
    create_schema: true
    logs_table_name: otel_logs
    traces_table_name: otel_traces
    metrics_table_name: otel_metrics
    ttl: 7d
    timeout: 5s
    retry:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 100s

Telemetry data types supported

logs, metrics, and traces

Is this a vendor-specific component?

[ ] This is a vendor-specific component
[ ] If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
[ ] If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

@xiaokang, @joker-star-l

Sponsor (optional)

@atoulme

Additional context

No response

joker-star-l commented 3 months ago

Hello @atoulme . @AlchemyDing talked to you about the Doris Exporter before. Could we invite you to be the sponsor?

AlchemyDing commented 3 months ago

Hello @atoulme , Do you have time to follow this issue, We need sponsors to help us complete this issue. Thank you.

atoulme commented 3 months ago

Yes, I can sponsor this component. The integration seems similar to the clickhouse exporter.

AlchemyDing commented 3 months ago

Yes, I can sponsor this component. The integration seems similar to the clickhouse exporter.

Thank you for your sponsorship. doris and clickhouse are both data warehouses, but doris exporter and clickhouse exporter have many differences in their export methods, such as the jdbc used by clickhouse exporter, the http used by doris exporter, and so on.

atoulme commented 3 months ago

Right - just pointing out I have had some experience helping on this type of exporter before. After that, it's all you.

To start, please follow the steps in CONTRIBUTING.md and prepare a first PR with a minimal skeleton for the exporter.

joker-star-l commented 3 months ago

Right - just pointing out I have had some experience helping on this type of exporter before. After that, it's all you.

To start, please follow the steps in CONTRIBUTING.md and prepare a first PR with a minimal skeleton for the exporter.

Thank you! I will submit the PRs according to the CONTRIBUTING.md.

AlchemyDing commented 3 months ago

We plan to have @joker-star-l responsible for the submission of PR, and I will help him in this process and then participate in the code review of his PR.

github-actions[bot] commented 1 month ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

open-telemetry / opentelemetry-collector-contrib