Tinyproxy 1.10 logs hostnames but 1.11 does not

[jsabater]

Tinyproxy version

I am using version 1.10.0-5 on Debian 11 Bullseye and version 1.11.1-2.1~bpo11+1 on Debian 12 Bookworm. In both cases LogLevel is set to Connect.

Issue

Tinyproxy 1.11.1 on Debian 12 Bookworm logs connections as follows:

CONNECT   Jul 19 09:02:02.529 [4102]: Connect (file descriptor 5): 192.168.0.119 
CONNECT   Jul 19 09:02:02.537 [4102]: Request (file descriptor 5): CONNECT ws.example.com:443 HTTP/1.0 
CONNECT   Jul 19 09:02:02.572 [4102]: Established connection to host "ws.example.com" using file descriptor 6. 

Tinyproxy 1.10.0 on Debian 11 Bullseye logs connections as follows:

CONNECT   Jul 19 09:03:32 [10103]: Connect (file descriptor 8): lxc35.domain.com [192.168.0.163]
CONNECT   Jul 19 09:03:32 [10103]: Request (file descriptor 8): CONNECT ws.example.com:443 HTTP/1.0
CONNECT   Jul 19 09:03:32 [10103]: Established connection to host "ws.example.com" using file descriptor 9.

Basically, Tinyproxy 1.11.1 no longer (resolves and) logs the hostnames.

I can think of this feature being removed (or disabled by default) in Tinyproxy 1.11.1 due to performance reasons, but I also would like to point out a difference in the setup of Tinyproxy 1.10: there is a PowerDNS recursor (resolver) listening on 127.0.0.1:53. Not sure if that would be detected and used by Tinyproxy 1.10.

Tinyproxy 1.11 has access to a DNS resolver, but it's in a separate container in the local network. DNS nameservers are configured and resolve requests correctly, though.

Is this intended? Is it possible to enable or disable it via the configuration file at /etc/tinyproxy/tinyproxy.conf?

Having the hostnames in the log files is very useful to me and, while I understand that it has a performance cost, it would be lovely for Tinyproxy to keep a cache of them and have the option to turn such feature on and off.

Thanks.

[rofl0r]

right, the reverse dns lookup caused considerable delays for every single connection. we got multiple complaints about this behaviour, but yours is the first complaint about the new behaviour. in theory, this could be easily fixed on your side by logging to stdout and piping the output through an awk script that replaces the ips in the respective connect lines with ip,hostname tuple like in 1.10. adding a config option to restore the previous behaviour is possible, though before going there i'd like to gather more opinions on the topic.

[jsabater]

I understand that it will have an impact on performance, hence the need to have an option in the configuration file (I really understand the reasoning behind the removal of this option).

Incidentally, did the code in version 1.10 cache the reverse DNS lookup responses, or did it query the DNS recursor each and every time? I ask because, in case it was querying the DNS recursor every time, in my opinion it would be a must to cache that information, perhaps also optionally. And, indeed, a DNS recursor in the localhost would help a lot, but that would be up to the user setting up Tinyproxy.

Thanks.

[rofl0r]

Incidentally, did the code in version 1.10 cache the reverse DNS lookup responses, or did it query the DNS recursor each and every time?

no, the people that added the lookup code weren't aware of the performance issue it causes. when it was brought to my attention, the obvious fix was "don't do it", not "add a cache". adding such a cache would be somewhat complex, as it would have to drop old items after a while in order to prevent ballooning of memory usage for the cache. imo this logic is better kept in an external tool. so the question is whether or not to add a config option to do the reverse lookup.

[jsabater]

I agree that adding a local DNS recursor (e.g. pdns_recursor) at 127.0.0.1:53 or similar may be a better way to support the option, in case it's enabled. :+1: