search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.13k stars 4.91k forks source link

Support receiving dnstap binary logs #12820

Open andrewkroh opened 5 years ago

andrewkroh commented 5 years ago

dnstap is binary log format that DNS servers support that records DNS requests and responses. This would allow Beats to receive DNS data directly from the DNS server rather than trying to get the information from packet capture. dnstap is supported by Bind, CoreDNS, Unbound, and more.

With new protocols like DNS-over-TLS (DoT) and DNS-over-HTTP (DoH) being deployed dnstap can be a necessary monitoring point for traffic that NSMs would otherwise not have visibility to.

An example deployment for an organization might be to have an internal DNS server providing UDP/TCP/DoT/DoH and blocking any DNS traffic (53/udp, 53/tcp, 853/tcp) direct to the internet. With dnstap being enabled, the server it would then let us monitor both the forwarded traffic from the server as well as all the traffic to/from clients.

I'm imagining a dnstap input type for Filebeat.

icb- commented 4 years ago

Is Filebeat the right place for this? I thought Filebeat was only for plain-text(-ish) formats, not pure binary formats like dnstap. Should it be its own separate Beat?

Large portions of the code could be lifted from the DNS protocol parser in Packetbeat. That seems to all be private functions. Should it be refactored into a supporting library that can be used by both Packetbeat and a "Dnstapbeat"? Keeping the format as similar as possible to the output of Packetbeat seems worthwhile.

lvg01 commented 2 years ago

I'm looking for this dnstap integration to the elastic-agent. Is there any progress on this? The packetbeat-like solution (with same ECS) would be a great solution!

botelastic[bot] commented 1 year ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

botelastic[bot] commented 1 year ago

This issue doesn't have a Team:<team> label.

icb- commented 1 year ago

Dnstap is definitely still wanted.

botelastic[bot] commented 8 months ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

lvg01 commented 8 months ago

Dnstap stays actual!