search

dmachard / go-dnscollector

Ingesting, pipelining, and enhancing your DNS logs with usage indicators, security analysis, and additional metadata.
MIT License
190 stars 44 forks source link

Feature request: extend Prometheus metrics by adding geoip statistics #376

Open KVInventoR opened 1 year ago

KVInventoR commented 1 year ago

Based on issue: https://github.com/dmachard/go-dnscollector/issues/370 I would like to request a new feature: support Geoip statistics over prometheus. it will be really useful to have:

based on this information, it will be much easier to identify DDOS attackers for DNS infrastructure

kabenin commented 1 year ago

Hi @dmachard , I was going to work on collecting stats per requester subnets for my needs. Will be happy to contribute to this request, if you are ok with that.

dmachard commented 1 year ago

@kabenin I missing time so it would be great. :)

johnhtodd commented 11 months ago

I'll throw another potential request in here since it is something we have already done in a separate pipeline (but would be happy to replace!) and while I suspect this is not something you'll implement entirely it's worth showing what we do so you can get some ideas for expanding this concept since you're already working on a patch. :-) We take a DNSTAP feed to produce this data.

Intead of "Top N" per ASN, it would be interesting to track everything. Yep, everything. We track all these stats, but if a prefix hasn't received any recent queries we delete all the metrics from memory. This actually works surprisingly well. We then fetch every minute or so, and get a very good idea of what's happening in the network and can historically examine networks shifts and DDoS issues.

There is an additional feature we have that adds a "country_code", "region" and "city_code" tag into each line (it does a lookup on an external database) but this doesn't work really well with BGP summarization so we don't use it much. If we were chunking things into /24 or /48 blocks then it would be more useful (and doing this at /32 or /128 is suicidal at least for us with the number of clients, and violates our privacy principles.) Very pretty maps can be made in Grafana showing all sorts of things.

We collect:

Per RCODE Per BGP announced prefix (or arbitrary CIDR cutoff)

Here's an example of actual output from our Go-based software (Prometheus format):

dnsanna_protocol{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",protocol="3",path_length="1"} 3 dnsanna_protocol{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",protocol="UDP",path_length="1"} 14946 dnsanna_qtype{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",qtype="AAAA",path_length="1"} 1926 dnsanna_qtype{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",qtype="A",path_length="1"} 10375 dnsanna_rcode{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",rcode="noerror",path_length="1"} 13741 dnsanna_rcode{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",rcode="nxdomain",path_length="1"} 1112 dnsanna_rcode{prefix="100.20.0.0/14",source="bgpd",origin_as="16509",transit_as="16509",tag="dnsdist",rcode="servfail",path_length="1"} 3

We're written this as a separate code base (https://indico.dns-oarc.net/event/27/contributions/478/ for a brief discussion by one of the authors.) I'm sure it could be woven into gocollector, but it would take a while. Happy to share the code if there is an interest in using some/all of it; we'd love to make it open-source and having it be in this codebase would be great.