DataDog / docker-dd-agent

Datadog Agent Dockerfile for Trusted Builds.
https://registry.hub.docker.com/u/datadog/docker-dd-agent/
MIT License
298 stars 189 forks source link

Datadog trace agent silently dies in dogstatsd container #309

Open anosulchik opened 6 years ago

anosulchik commented 6 years ago

Hi, we run datadog/docker-dd-agent:latest-dogstatsd docker container configured with option DD_APM_ENABLED=yes so it should accept traces from dd-trace-js. Soon after container is started it works as expected - it sends traces which are available at Datadog APM interface, we also see the following in the logs until port 8126 becomes unavailable in container and traces stops:

2018-06-14 14:25:12 INFO (receiver.go:325) - [lang:nodejs lang_version:v8.11.2 interpreter:v8 tracer_version:0.2.1] -> traces received: 32, traces dropped: 0, traces filtered: 0, traces
 amount: 18910 bytes, services received: 0, services amount: 0 bytes
2018-06-14 14:25:12 INFO (stats_writer.go:73) - flushed stat payload to the API, time:3.160773ms, size:891 bytes
2018-06-14 14:25:12 INFO (trace_writer.go:89) - flushed trace payload to the API, time:8.344897ms, size:684 bytes
2018-06-14 14:25:17 INFO (trace_writer.go:89) - flushed trace payload to the API, time:7.217358ms, size:907 bytes
2018-06-14 14:25:22 INFO (stats_writer.go:73) - flushed stat payload to the API, time:3.159326ms, size:891 bytes
2018-06-14 14:25:22 INFO (trace_writer.go:89) - flushed trace payload to the API, time:4.601284ms, size:683 bytes
2018-06-14 14:25:32 INFO (stats_writer.go:73) - flushed stat payload to the API, time:5.345281ms, size:1114 bytes
2018-06-14 14:25:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:26:22 INFO (receiver.go:325) - [lang:nodejs lang_version:v8.11.2 interpreter:v8 tracer_version:0.2.1] -> traces received: 6, traces dropped: 0, traces filtered: 0, traces
amount: 3190 bytes, services received: 0, services amount: 0 bytes
2018-06-14 14:26:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:27:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:28:32 INFO (receiver.go:325) - no data received
2018-06-14 14:28:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:29:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:29:42 INFO (receiver.go:325) - no data received
2018-06-14 14:30:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:30:52 INFO (receiver.go:325) - no data received
2018-06-14 14:31:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:31:52 INFO (receiver.go:325) - no data received
2018-06-14 14:32:42 INFO (service_mapper.go:59) - total number of tracked services: 2
2018-06-14 14:33:02 INFO (receiver.go:325) - no data received

Please notice that 2018-06-14 14:25:42 traces stopped for unknown reason. The only way to "fix" this is to restart container. Please advise.