Transform domain names, before processing them

[KVInventoR]

Hi @dmachard

Thank you for this wonderful project, I see how things were changed and this is really wonderful. I remember that, for the python version of collector, I asked to apply lower-case filter for domain names before processing them, right now, I found in Prometheus statistics, that there are multiple domains with mixed cases. Probably, you already resolved this issue, but unfortunately, I didn't find anything related in the changelog

[KVInventoR]

Just checked my config and found:

global:
  trace:
    verbose: true
    log-malformed: true

multiplexer:
  collectors:
    - name: tap
      powerdns:
        listen-ip: 0.0.0.0
        listen-port: 6000
      transforms:
        normalize:
          lowercase-qname: true
          add-tld: true
          add-tld-plus-one: true
        suspicious:
          enable: true
        public-suffix:
          add-tld: true
        geoip:
          mmdb-country-file: "/GeoLite2-Country.mmdb"
          mmdb-city-file: ""
          mmdb-asn-file: ""

generally, something wrong only in section:

HELP dnscollector_top_sfdomains Number of hit per servfail domain topN, partitioned by qname

TYPE dnscollector_top_sfdomains gauge

probably, transformation doesn't apply here

[dmachard]

The transform lowercase is applied on collector side then the qname is send to loggers in lowercase so in fact it should work as you expect.

I will try to reproduce and come back to you

[dmachard]

Can you share more details, the complete config file ? I did more tests without reproducing your behavior.

[KVInventoR]

Hi, this is full my config

global:
  trace:
    verbose: true
    log-malformed: true

multiplexer:
  collectors:
    - name: tap
      powerdns:
        listen-ip: 0.0.0.0
        listen-port: 6000
      transforms:
        normalize:
          lowercase-qname: true
          add-tld: true
          add-tld-plus-one: true
        suspicious:
          enable: true
        public-suffix:
          add-tld: true
        geoip:
          mmdb-country-file: "/GeoLite2-Country.mmdb"
          mmdb-city-file: ""
          mmdb-asn-file: ""

  loggers:
    - name: console
      stdout:
        mode: text

    - name: json
      logfile:
        file-path:  /var/dnscollector/dnstap.log
        mode: json
        max-size: 3
        max-files: 10

    - name: api
      restapi:
        listen-ip: 0.0.0.0
        listen-port: 8080

    - name: prom
      prometheus:
        listen-ip: 0.0.0.0
        listen-port: 8081
        top-n: 50

    - name: loki
      lokiclient:
        server-url: "http://loki-gateway:3100/loki/api/v1/push"
        job-name: "dnscollector"
        mode: "json"
        flush-interval: 5
        batch-size: 1048576
        retry-interval: 10
  #      text-format: "localtime identity qr queryip family protocol qname qtype rcode"
        proxy-url: ""
        tls-insecure: false
        tls-min-version: 1.2
        basic-auth-login: ""
        basic-auth-pwd: ""
        tenant-id: "docker"

  routes:
    - from: [ tap ]
      to: [ console, json, api, prom, loki ]

[dmachard]

Thanks, I found the mistake in your config on the key name: lowercase-qname Replaces it by qname-lowercase It should be ok :)

[KVInventoR]

I think, needs to be fixed also here: https://github.com/dmachard/go-dnscollector/blob/4e9f5f602898b19509a84801e85a3a238cae600a/doc/docker.md?plain=1#L47