zmoog
commented
6 months ago OTel Collector
I need to learn how to set up and run the OTel Collector first.
Overview
Since the Elastic Stack natively supports the OpenTelemetry protocol (OTLP), we will send the metrics directly to the Elastic Stack.
Deployment model
I will use Docker and Docker Compose as deployment method, using https://github.com/open-telemetry/opentelemetry-demo/blob/main/docker-compose.minimal.yml as template.
Docker Compose
We'll use the otel/opentelemetry-collector-contrib:0.91.0 image and a volume to provide a custom config file named ./src/otelcollector/otelcol-config.yml.
# docker-compose.yml
services:
# OpenTelemetry Collector
otelcol:
image: otel/opentelemetry-collector-contrib:0.91.0
container_name: otel-col
deploy:
resources:
limits:
memory: 125M
restart: unless-stopped
command: [ "--config=/etc/otelcol-config.yml" ]
volumes:
- ./src/otelcollector/otelcol-config.yml:/etc/otelcol-config.yml
ports:
- "4317" # OTLP over gRPC receiver
- "4318" # OTLP over HTTP receiver
environment:
- ELASTIC_APM_SERVER_ENDPOINT="<DEPLOYMENTapm.eastus2.azure.elastic-cloud.com:443"
- ELASTIC_APM_SECRET_TOKEN="<TOKEN>" OTel Collector config file
We are setting up the OTel collector to pull metrics from Azure Monitor API and send them to Elasticsearch using the OTLP on the APM server.
Here's the config file:
receivers:
otlp:
protocols:
grpc:
http:
cors:
allowed_origins:
- "http://*"
- "https://*"
azuremonitor:
subscription_id: "${subscription_id}"
tenant_id: "${tenant_id}"
client_id: "${client_id}"
client_secret: "${client_secret}"
cloud: AzureCloud
# resource_groups:
# - "${resource_group1}"
# - "${resource_group2}"
services:
- "Microsoft.Compute/virtualMachines"
collection_interval: 60s
initial_delay: 1s
exporters:
logging:
loglevel: warn
otlp/elastic:
# Elastic APM server https endpoint without the "https://" prefix
endpoint: "${ELASTIC_APM_SERVER_ENDPOINT}"
headers:
# Elastic APM Server secret token
Authorization: "Bearer ${ELASTIC_APM_SECRET_TOKEN}"
service:
pipelines:
metrics:
receivers: [otlp, azuremonitor]
exporters: [logging, otlp/elastic]
I want to collect metric values using the azuremonitorreceiver from the opentelemetry-collector-contrib project.
The goal is evaluate how it works while collecting PT5M metrics using a 5 mins collection interval.