Closed pfsiedl closed 6 months ago
Hi how old is your nginx setup? fingerprints db looks quiet oversized to me
Hello, the creation time of handshakes.json is October 2021, so quite old.
Should we prune this file via cronjob every now and then?
@pfsiedl There were changes in the tls_ua_fingerprint.php
that were supposed to reduce the number of unique fingerprints received. but that was not that long ago)
I think you can just delete the handshakes.json
file, it will start being created again automatically
@kulikov-a I've deleted the handshakes.json file and let it rebuild. Seems to work for the time being.
Is there any description behind the mechanism of handshakes.json / tls_handshakes and why the plugin stores them separately in a json file? Is this only for visuals and ban-features via GUI?
@pfsiedl the general idea is described in the docs (https://docs.opnsense.org/manual/how-tos/nginx_tls_fingerprints.html)
so tls_handshake.log
is a log created by the nginx and containing the agent name, ciphers and curves. Once every 10 minutes, the data in this log is merged with the data in the permanent fingerprint db (handshakes.json
).
You can see handshakes.json content at the SERVICES: NGINX: TLS FINGERPRINTS and use it to save the agent’s fingerprint and try to control possible traffic interception (if the fingerprint in request differs from the reference one). X-TLS-Client-Intercepted header is used to pass this info to the upstream
@kulikov-a we resolved the issue with cleanup of handshakes.json.
We again running into the issue, that handshakes.json file is quite large and therefore the php process runs into an oom exception. Current size if handshakes.json is 259MB.
Can't we have this file deleted once a week using a cronjob until you implement a longer-term fix? Can I provide you with more information?
hi! I'll try to take a look at the end of the week. a little busy right now, sorry
@pfsiedl Hi! Sorry, I don't know the devs' plans for merging the previous pr (https://github.com/opnsense/plugins/pull/3678). Therefore, I don’t understand whether it makes sense to try to add something there (for example, the ability to disable handshake logging for the server (there is a feeling that few people use the ability to use fingerprints for mitm detect. and adding the ability to disable it would be appropriate)). In the meantime, I think you could delete this file yourself or add a configuration file (like http://kb.unixservertech.com/other/networking/opnsense/cron-jobs) that allows you to add a cron job through the GUI (tell me if you need help with this file). However, i think that the handshakes db file size seems unusually large to me (on my production setup with a dozen sites, the file size does not exceed several megabytes. and I haven't encountered any complaints about this issue before). It would be interesting to look at the contents of your file - perhaps this will help us understand the root of the problem.
@kulikov-a I think I've found the culprit. We are using a groupware that alters the User-Agent header every request from their application to the server, and thats proxied via nginx.
The creation of the cronjob worked flawless and is a good workaround.
We would really appreciate an option to disable the handshake logging per server and hope your PR gets accepted asap to continue working on this plugin. Thank you for your effort.
@pfsiedl Glad to hear that you managed to resolve the issue and imho this is definitely a reason to add a 'Log TLS handshake' checkbox. I'll try to add this to the request)
Important notices Before you add a new report, we ask you kindly to acknowledge the following:
Describe the bug PHP process crashes due to excessive memory usage when writing the file /var/log/nginx/handshakes.json by the script tls_ua_fingerprint.php.
To Reproduce Not a clear solution to reproduce the behavior, just some facts. /var/log/nginx/handshakes.json got a filesize of 234MB. According to the traffic statistics, there have been over 1,4 Mio. requests in 20 days uptime, of which 320.000 requests have been accepted.
Expected behavior No crash due to an more memory efficient way to write the file.
Relevant log files
[21-Feb-2024 22:20:02 Europe/Vienna] PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 216010752 bytes) in /usr/local/opnsense/scripts/nginx/tls_ua_fingerprint.php on line 135
Additional context We already have two independent firewalls with crashes of this type. If you need any further logs or informations, I try my best to provide you with the additional informations.
Environment