open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.02k stars 2.33k forks source link

Bare minimum docker-compose sample for EXPORTER/LOKI #29340

Closed cilerler closed 11 months ago

cilerler commented 11 months ago

Component(s)

exporter/loki

Describe the issue you're reporting

I am trying to export logs through OTEL-COLLECTOR to LOKI via the LOKI-EXPORTER. With the setup below, I see the logs in the OTEL-COLLECTOR container, but not in LOKI. As far as I can tell, there is no immediate visible error. I would thoroughly appreciate any help to point me in the right direction to make this work.

docker-compose.yml

```yaml version: '3.8' networks: default: name: my_network driver: bridge services: otel-collector: container_name: otel-collector image: otel/opentelemetry-collector-contrib:0.88.0 restart: always command: ["--config=/etc/otel-collector-config.yaml", ""] volumes: - ./config/otel-collector.yaml:/etc/otel-collector-config.yaml ports: - 1888 # pprof extension - 8888 # Prometheus metrics exposed by the Collector - 8889 # Prometheus exporter metrics - 13133 # health_check extension - 4317:4317 # OTLP gRPC receiver - 4318:4318 # OTLP http receiver - 55679 # zpages extension depends_on: - loki minio: container_name: minio image: minio/minio:latest entrypoint: - sh - -euc - | mkdir -p /data/loki-data && \ mkdir -p /data/loki-ruler && \ mkdir -p /data/tempo && \ minio server /data \ --console-address ':9001' environment: - MINIO_ROOT_USER=minio - MINIO_ROOT_PASSWORD=minio123 - MINIO_PROMETHEUS_AUTH_TYPE=public ports: - 9001 loki: container_name: loki image: grafana/loki:latest command: "-config.file=/etc/loki/config.yaml" ports: - 3100 - 7946 - 9095 volumes: - ./config/loki.yml:/etc/loki/config.yaml depends_on: - minio grafana: container_name: grafana image: grafana/grafana:latest environment: - GF_PATHS_PROVISIONING=/etc/grafana/provisioning - GF_AUTH_ANONYMOUS_ENABLED=true - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin - GF_AUTH_DISABLE_LOGIN_FORM=true - GF_FEATURE_TOGGLES_ENABLE=traceqlEditor ports: - "3000:3000" volumes: - ./config/grafana-datasources.yml:/etc/grafana/provisioning/datasources/ds.yaml depends_on: - loki ```

config/otel-collector.yml

```yaml receivers: otlp: protocols: grpc: exporters: debug: verbosity: detailed loki: endpoint: http://loki:3100/loki/api/v1/push tls: insecure: true headers: "X-Scope-OrgID": "1" default_labels_enabled: exporter: true job: true instance: true level: true processors: batch: attributes: actions: - action: insert key: loki.attribute.labels value: test resource: attributes: - action: insert key: loki.resource.labels value: test service: pipelines: logs: receivers: [otlp] processors: [batch, resource, attributes] exporters: [debug, loki] ```

config/loki.yml

```yaml server: http_listen_port: 3100 schema_config: configs: - from: 2021-08-01 store: boltdb-shipper object_store: s3 schema: v11 index: prefix: index_ period: 24h common: path_prefix: /loki replication_factor: 1 storage: s3: endpoint: minio:9000 insecure: true bucketnames: loki-data access_key_id: minio secret_access_key: minio123 s3forcepathstyle: true ring: kvstore: store: memberlist ruler: storage: s3: bucketnames: loki-ruler ```

config/grafana-datasource.yml

```yml apiVersion: 1 datasources: - name: Loki uid: loki type: loki isDefault: true version: 1 editable: false access: proxy url: http://loki:3100 jsonData: derivedFields: - datasourceUid: tempo matcherRegex: "TraceId=(\\w+)" name: TraceId url: "$${__value.raw}" httpHeaderName1: "X-Scope-OrgID" secureJsonData: httpHeaderValue1: "tenant1" ```

github-actions[bot] commented 11 months ago

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

jpkrohling commented 11 months ago

"X-Scope-OrgID": "1"

httpHeaderValue1: "tenant1"

Looks like the tenant's aren't matching in the two configs?

cilerler commented 11 months ago

You made my day, thank you very much.

jpkrohling commented 11 months ago

And you've just made mine :-)

skumarjh-tibco commented 8 months ago

I am trying to do the setup with same files. Can you elaborate what was missing ? Thanks!

cilerler commented 8 months ago
    secureJsonData:
      httpHeaderValue1: "tenant1"

in config/otel-collector.yml, and

    headers:
      "X-Scope-OrgID": "1"

in config/grafana-datasource.yml

values have to match.

in other words the issue was "tenant1" != "1"

skumarjh-tibco commented 8 months ago

Thanks a lot! It worked.