search

DataDog / datadog-agent

Main repository for Datadog Agent
https://docs.datadoghq.com/
Apache License 2.0
2.89k stars 1.21k forks source link

Logs Agent is not running #9373

Closed realtebo closed 3 years ago

realtebo commented 3 years ago

Output of the info page (if this is a bug)

===============
Agent (v7.31.1)
===============

  Status date: 2021-10-04 18:19:59.153 CEST / 2021-10-04 16:19:59.153 UTC (1633364399153)
  Agent start: 2021-10-04 18:10:40.918 CEST / 2021-10-04 16:10:40.918 UTC (1633363840918)
  Pid: 406821
  Go Version: go1.15.13
  Python Version: 3.8.11
  Build arch: amd64
  Agent flavor: agent
  Check Runners: 4
  Log Level: info

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: 269µs
    System time: 2021-10-04 18:19:59.153 CEST / 2021-10-04 16:19:59.153 UTC (1633364399153)

  Host Info
  =========
    bootTime: 2021-08-16 13:06:20 CEST / 2021-08-16 11:06:20 UTC (1629111980000)
    kernelArch: x86_64
    kernelVersion: 5.10.0-8-amd64
    os: linux
    platform: debian
    platformFamily: debian
    platformVersion: 11.0
    procs: 124
    uptime: 1181h4m28s
    virtualizationRole: guest

  Hostnames
  =========
    hostname: li1657-192
    socket-fqdn: li1657-192.members.linode.com.
    socket-hostname: li1657-192
    hostname provider: os
    unused hostname providers:
      aws: not retrieving hostname from AWS: the host is not an ECS instance and other providers already retrieve non-default hostnames
      azure: azure_hostname_style is set to 'os'
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: Get "http://169.254.169.254/computeMetadata/v1/instance/hostname": dial tcp 169.254.169.254:80: i/o timeout (Client.Timeout exceeded while awaiting headers)

  Metadata
  ========
    hostname_source: os

=========
Collector
=========

  Running Checks
  ==============

    cpu
    ---
      Instance ID: cpu [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 9, Total: 326
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-10-04 18:19:45 CEST / 2021-10-04 16:19:45 UTC (1633364385000)
      Last Successful Execution Date : 2021-10-04 18:19:45 CEST / 2021-10-04 16:19:45 UTC (1633364385000)

    disk (4.4.0)
    ------------
      Instance ID: disk:e5dffb8bef24336f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/disk.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 56, Total: 2,072
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 12ms
      Last Execution Date : 2021-10-04 18:19:53 CEST / 2021-10-04 16:19:53 UTC (1633364393000)
      Last Successful Execution Date : 2021-10-04 18:19:53 CEST / 2021-10-04 16:19:53 UTC (1633364393000)

    file_handle
    -----------
      Instance ID: file_handle [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
      Total Runs: 36
      Metric Samples: Last Run: 5, Total: 180
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-10-04 18:19:44 CEST / 2021-10-04 16:19:44 UTC (1633364384000)
      Last Successful Execution Date : 2021-10-04 18:19:44 CEST / 2021-10-04 16:19:44 UTC (1633364384000)

    io
    --
      Instance ID: io [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/io.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 26, Total: 944
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-10-04 18:19:51 CEST / 2021-10-04 16:19:51 UTC (1633364391000)
      Last Successful Execution Date : 2021-10-04 18:19:51 CEST / 2021-10-04 16:19:51 UTC (1633364391000)

    load
    ----
      Instance ID: load [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/load.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 6, Total: 222
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-10-04 18:19:58 CEST / 2021-10-04 16:19:58 UTC (1633364398000)
      Last Successful Execution Date : 2021-10-04 18:19:58 CEST / 2021-10-04 16:19:58 UTC (1633364398000)

    memory
    ------
      Instance ID: memory [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/memory.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 18, Total: 666
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-10-04 18:19:50 CEST / 2021-10-04 16:19:50 UTC (1633364390000)
      Last Successful Execution Date : 2021-10-04 18:19:50 CEST / 2021-10-04 16:19:50 UTC (1633364390000)

    network (2.3.0)
    ---------------
      Instance ID: network:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/network.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 31, Total: 1,147
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 3ms
      Last Execution Date : 2021-10-04 18:19:57 CEST / 2021-10-04 16:19:57 UTC (1633364397000)
      Last Successful Execution Date : 2021-10-04 18:19:57 CEST / 2021-10-04 16:19:57 UTC (1633364397000)

    ntp
    ---
      Instance ID: ntp:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
      Total Runs: 1
      Metric Samples: Last Run: 1, Total: 1
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 1
      Average Execution Time : 41ms
      Last Execution Date : 2021-10-04 18:10:46 CEST / 2021-10-04 16:10:46 UTC (1633363846000)
      Last Successful Execution Date : 2021-10-04 18:10:46 CEST / 2021-10-04 16:10:46 UTC (1633363846000)

    uptime
    ------
      Instance ID: uptime [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
      Total Runs: 37
      Metric Samples: Last Run: 1, Total: 37
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-10-04 18:19:49 CEST / 2021-10-04 16:19:49 UTC (1633364389000)
      Last Successful Execution Date : 2021-10-04 18:19:49 CEST / 2021-10-04 16:19:49 UTC (1633364389000)

========
JMXFetch
========

  Information
  ==================
  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    Cluster: 0
    CronJob: 0
    DaemonSet: 0
    Deployment: 0
    Dropped: 0
    DroppedOnInput: 0
    Job: 0
    Node: 0
    PersistentVolume: 0
    PersistentVolumeClaim: 0
    Pod: 0
    ReplicaSet: 0
    Requeued: 0
    Retried: 0
    RetryQueueSize: 0
    Service: 0
    StatefulSet: 0

  Transaction Successes
  =====================
    Total number: 78
    Successes By Endpoint:
      check_run_v1: 37
      intake: 4
      series_v1: 37

  API Keys status
  ===============
    API key ending with ee819: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.eu - API Key ending with:
      - ee819

==========
Logs Agent
==========

  Logs Agent is not running

=========
APM Agent
=========
  Status: Running
  Pid: 406823
  Uptime: 558 seconds
  Mem alloc: 7,897,944 bytes
  Hostname: li1657-192
  Receiver: localhost:8126
  Endpoints:
    https://trace.agent.datadoghq.eu

  Receiver (previous minute)
  ==========================
    No traces received in the previous minute.
    Default priority sampling rate: 100.0%

  Writer (previous minute)
  ========================
    Traces: 0 payloads, 0 traces, 0 events, 0 bytes
    Stats: 0 payloads, 0 stats buckets, 0 bytes

=========
Aggregator
=========
  Checks Metric Sample: 6,187
  Dogstatsd Metric Sample: 4,380
  Event: 1
  Events Flushed: 1
  Number Of Flushes: 37
  Series Flushed: 7,077
  Service Check: 297
  Service Checks Flushed: 332
=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 4,379
  Metric Parse Errors: 0
  Service Check Packets: 0
  Service Check Parse Errors: 0
  Udp Bytes: 335,651
  Udp Packet Reading Errors: 0
  Udp Packets: 2,967
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0
  Unterminated Metric Errors: 0

root@li1657-192:~# systemctl restart datadog-agent
root@li1657-192:~# sudo datadog-agent status
Getting the status from the agent.

template: /trace-agent.tmpl:22:15: executing "/trace-agent.tmpl" at <len .receiver>: error calling len: len of nil pointer
===============
Agent (v7.31.1)
===============

  Status date: 2021-10-04 18:20:26.876 CEST / 2021-10-04 16:20:26.876 UTC (1633364426876)
  Agent start: 2021-10-04 18:20:18.114 CEST / 2021-10-04 16:20:18.114 UTC (1633364418114)
  Pid: 407913
  Go Version: go1.15.13
  Python Version: 3.8.11
  Build arch: amd64
  Agent flavor: agent
  Check Runners: 4
  Log Level: info

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    System time: 2021-10-04 18:20:26.876 CEST / 2021-10-04 16:20:26.876 UTC (1633364426876)

  Host Info
  =========
    bootTime: 2021-08-16 13:06:20 CEST / 2021-08-16 11:06:20 UTC (1629111980000)
    kernelArch: x86_64
    kernelVersion: 5.10.0-8-amd64
    os: linux
    platform: debian
    platformFamily: debian
    platformVersion: 11.0
    procs: 128
    uptime: 1181h14m3s
    virtualizationRole: guest

  Hostnames
  =========
    hostname: li1657-192
    socket-fqdn: li1657-192.members.linode.com.
    socket-hostname: li1657-192
    hostname provider: os
    unused hostname providers:
      aws: not retrieving hostname from AWS: the host is not an ECS instance and other providers already retrieve non-default hostnames
      azure: azure_hostname_style is set to 'os'
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: Get "http://169.254.169.254/computeMetadata/v1/instance/hostname": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

  Metadata
  ========

=========
Collector
=========

  Running Checks
  ==============
    No checks have run yet

========
JMXFetch
========

  Information
  ==================
  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    Cluster: 0
    CronJob: 0
    DaemonSet: 0
    Deployment: 0
    Dropped: 0
    DroppedOnInput: 0
    Job: 0
    Node: 0
    PersistentVolume: 0
    PersistentVolumeClaim: 0
    Pod: 0
    ReplicaSet: 0
    Requeued: 0
    Retried: 0
    RetryQueueSize: 0
    Service: 0
    StatefulSet: 0

  API Keys status
  ===============
    API key ending with ee819: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.eu - API Key ending with:
      - ee819

==========
Logs Agent
==========

  Logs Agent is not running

=========
APM Agent
=========
  Status: Running
  Pid: 407915
  Uptime: 8 seconds
  Mem alloc: 7,754,384 bytes
  Hostname: li1657-192
  Receiver: localhost:8126
  Endpoints:
    https://trace.agent.datadoghq.eu

  Receiver (previous minute)
  ===================================
Aggregator
=========
  Dogstatsd Metric Sample: 1
  Event: 1
=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 0
  Metric Parse Errors: 0
  Service Check Packets: 0
  Service Check Parse Errors: 0
  Udp Bytes: 0
  Udp Packet Reading Errors: 0
  Udp Packets: 0
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0
  Unterminated Metric Errors: 0

Describe what happened:

On a Debian 11, I installed the agent, and configured to enble log collecting and then I added a nano /etc/datadog-agent/conf.d/nginx.d/conf.yaml

at the end I have

logs:
  - type: file
    path: /var/log/nginx/*.log
    source: nginx
    service: <name of my web app>

Describe what you expected:

I expected to see logs in the web dashboard

Additional environment details (Operating System, Cloud provider, etc):

Debian 11 VPs on Linode Cloud

MaiBAlmutairi commented 2 years ago

I'm having the same problem how did you solve it ?

bradq commented 2 years ago

@MaiBAlmutairi likely a bit late for you, but logging intake is not enabled on the agent by default. logs_enabled:true, or the same in envvars/Docker labels/K8s annotations will do the trick.

ics-ajay commented 7 months ago

I am also facing this issue, even after making this change logs_enabled:true