Include requester ip address in ForwardHandler received request debug log

gberche-orange commented 2 years ago

Expected behavior

As a bosh-dns user

in order to easily identify DNS client that make unexpected DNS request to a bosh-dns process
I need the IP address of the remote machine that made this request

Observed behavior

The logs don't yet include the source IP address of the received DNS request

[ForwardHandler] 2021-10-15T12:53:07.404484142Z DEBUG - handlers.ForwardHandler Received request id=14476 qtype=[AAAA] qname=[api.github.com.]
[ForwardHandler] 2021-10-15T12:53:07.407406989Z DEBUG - handlers.ForwardHandler Received request id=30384 qtype=[A] qname=[api.github.com.]

Context

A bosh-dns job is acting as a DNS resolver for multiple client vms. Client vms are expected to use http proxy for internet traffic and only resolve intranet FQDNs through the bosh-dns resolver. Identifying client vms that incorrectly request internet FQDNs is time consumming.

Workaround

Use tcpdump traces

bosh-admin-bot commented 2 years ago

This issue was marked as Stale because it has been open for 21 days without any activity. If no activity takes place in the coming 7 days it will automatically be close. To prevent this from happening remove the Stale label or comment below.

bgandon commented 2 years ago

Hi Guillaume! It seems that the work submitted by SAP in #78 could be a good base for your need. You would just have to add the source IP address to the produced log lines. Would that fit your requirement?

gberche-orange commented 2 years ago

@bgandon The requirement to log the source ip address seems independent to me of the logging configuration options. I'm however happy if this ends up being bundled within a PR offering a new "recursion-logging" opt-in

beyhan commented 2 years ago

Hi @gberche-orange,

I was trying to understand when this additional information can help but I can't find an useful use case. Usually, you have following two situations to look into bosh-dns logs

on a bosh VM itself and there the request ip is clear
bosh-dns logs forwarded to a logging service. In this case the forwarder can enrich the log entries with the VM ip from where the logs are coming

Do you have a different use case?

bosh-admin-bot commented 2 years ago

This issue was marked as Stale because it has been open for 21 days without any activity. If no activity takes place in the coming 7 days it will automatically be close. To prevent this from happening remove the Stale label or comment below.

bosh-admin-bot commented 2 years ago

This issue was closed because it has been labeled Stale for 7 days without subsequent activity. Feel free to re-open this issue at any time by commenting below.

bgandon commented 2 years ago

But wait a minute, is a Bosh-DNS job really able to act as a DNS resolver for multiple client VMs?

Possibly you've made an invalid assumption in the issue description above @gberche-orange, because Bosh DNS server is only available for queries performed on localhost. This means that when you see a DNS query log line, then you don't need the IP address of any remote client that has made the request. The client cannot be remote. That IP address can only be 169.254.0.2, so you can conclude that a process on the localhost is responsible for sending the DNS query.

Indeed, when you inspect the TCP/UDP sockets on some Bosh instance, the DNS server is only bound to the 169.254.0.2 IP address as shown below:

# netstat -ntulp | sort -n -t: -k2
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      151/sshd            
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
tcp        0      0 169.254.0.2:53          0.0.0.0:*               LISTEN      2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
udp        0      0 169.254.0.2:53          0.0.0.0:*                           2125/bosh-dns       
tcp        0      0 127.0.0.1:2822          0.0.0.0:*               LISTEN      2178/monit-actual   
tcp        0      0 127.0.0.1:2825          0.0.0.0:*               LISTEN      56/bosh-agent       
tcp        0      0 0.0.0.0:8853            0.0.0.0:*               LISTEN      2101/bosh-dns-healt 
tcp        0      0 127.0.0.1:53080         0.0.0.0:*               LISTEN      2125/bosh-dns

Plus, the 169.254.0.2 is assigned to the loopback lo network device, as show below:

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 169.254.0.2/32 scope global lo
       valid_lft forever preferred_lft forever
...

Having said that, my understanding could possibly be wrong, or you could also need to track bad clients that manage to send DNS queries remotely for some reasons that I'm not aware of.

Guillaume, could you detail further your requirement considering the details shown above?

bosh-admin-bot commented 2 years ago

This issue was marked as Stale because it has been open for 21 days without any activity. If no activity takes place in the coming 7 days it will automatically be close. To prevent this from happening remove the Stale label or comment below.

bosh-admin-bot commented 2 years ago

This issue was closed because it has been labeled Stale for 7 days without subsequent activity. Feel free to re-open this issue at any time by commenting below.

poblin-orange commented 1 year ago

@bgandon @beyhan missed to follow up. In our use case, we use bosh-dns exposed publicy, to enable bosh dns resolution from other directors (we have chained architecture micro-bosh => bosh master).

cloudfoundry / bosh-dns-release