dmachard / go-dnscollector

Ingesting, pipelining, and enhancing your DNS logs with usage indicators, security analysis, and additional metadata.
MIT License
210 stars 47 forks source link

deadlock situations may occur with v0.40.0 - regression #568

Closed denis-machard closed 9 months ago

denis-machard commented 9 months ago

Describe the bug

Jan 23 21:13:56 tapx go-dnscollector[43716]: INFO: 2024/01/23 21:13:56.462172 [fileA] logger=file - execute compress postrotate command: /var/tap/dnstap-1706040822558956943.log.gz
Jan 23 21:14:04 tapx go-dnscollector[43716]: ERROR: 2024/01/23 21:14:04.747554 [tapA] collector=powerdns - recv buffer is full, 136058 packet(s) dropped
Jan 23 21:14:14 tapx go-dnscollector[43716]: ERROR: 2024/01/23 21:14:14.749079 [tapA] collector=powerdns - recv buffer is full, 164877 packet(s) dropped
Jan 23 21:14:24 tapx go-dnscollector[43716]: ERROR: 2024/01/23 21:14:24.749696 [tapA] collector=powerdns - recv buffer is full, 158886 packet(s) dropped

Jan 24 10:22:54 tapx go-dnscollector[54904]: INFO: 2024/01/24 10:22:54.432103 [fileA] transformer=reducer#0 - enabled

global:
  trace:
    verbose: true

multiplexer:
  collectors:
    - name: tapIN
      powerdns:
        listen-ip: 127.0.0.1
        listen-port: 6000
      transforms:
        normalize:
          qname-lowercase: true
        latency:
          measure-latency: true

  loggers:
    - name: fileA
      logfile:
        file-path:  "/var/tap/dnstap.log"
        max-size: 250
        max-files: 10
        mode: text
        compress: true
        compress-postcommand: "/var/tap/postrotate.sh"
        text-format: "localtime identity operation queryip family protocol qname qtype rcode latency reducer-occurences"
      transforms:
        reducer:
          repetitive-traffic-detector: true
          watch-interval: 5

    - name: prom
      prometheus:
        listen-ip: 0.0.0.0
        listen-port: 8080
        top-n: 10
      transforms:
        filtering:
          log-queries: false

  routes:
    - from: [ tapIN  ]
      to: [ fileA, prom ]
dmachard commented 9 months ago

regression related to https://github.com/dmachard/go-dnscollector/issues/521