open-telemetry / opentelemetry-collector

OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
4.32k stars 1.43k forks source link

[confighttp] Make the compression level configurable #10467

Open rnishtala-sumo opened 3 months ago

rnishtala-sumo commented 3 months ago

Current compression algorithms seem to support multiple compression levels. These levels determine the speed and efficiency of the compression algorithm.

Lower compression levels provide faster speed but larger file sizes. Compression levels could be 3, 5, 7 or 11. Level 3 seems to be the default across multiple compression algorithms. It could benefit a user to give them the option to save on AWS egress costs by configuring a higher compression level with a tradeoff on processing speed.

Example configuration could looks like below:

exporter:
  otlphttp:
    endpoint: otelcol2:55690
    auth:
      authenticator: some-authenticator-extension
    tls:
      ca_file: ca.pem
      cert_file: cert.pem
      key_file: key.pem
    headers:
      test1: "value1"
      "test 2": "value 2"
    compression
       type: zstd
       level: 7
    cookies:
      enabled: true
rnishtala-sumo commented 3 months ago

I like the idea of configuring this as a single string

exporter:
  otlphttp:
    endpoint: otelcol2:55690
    auth:
      authenticator: some-authenticator-extension
    tls:
      ca_file: ca.pem
      cert_file: cert.pem
      key_file: key.pem
    headers:
      test1: "value1"
      "test 2": "value 2"
    compression: zstd/7
    cookies:
      enabled: true
wasim-nihal commented 2 months ago

Hi @mx-psi /@rnishtala-sumo , is this open for contribution?

rnishtala-sumo commented 2 months ago

@wasim-nihal yes, please proceed if this issue is of interest to you. I can assist with the PR review.

rnishtala-sumo commented 1 month ago

@mx-psi @dmitryax if there's no one working on this, could this issue be assigned to me?

mx-psi commented 1 month ago

Assigned!

rnishtala-sumo commented 2 weeks ago

@dmitryax @mx-psi I've raised a PR for this issue. Would appreciate your review.