search

xavier-hernandez / goaccess-for-nginxproxymanager

GoAccess Docker Image for Nginx Proxy Manager and more...
https://hub.docker.com/r/xavierh/goaccess-for-nginxproxymanager
MIT License
532 stars 39 forks source link

Add default-host, dead-host, and fallback to selectable NPM options. #136

Closed DerLeole closed 1 year ago

DerLeole commented 1 year ago

I originally installed this so I could see which subdomains that I have not actually assigned would be attempted to be reached by others, to identify crawlers and/or attempts to comb the domain for open services.

Sadly the default NPM settings entirely ignore fallback_access.log, default-host_access.log and dead-host-1_access.log, which seems to split these "lost" requests between them.

Since they are part of the default NPM log package and provide very valuable data, I think they would be a great option for either an additional NPM setting with a subdomain like /misses or added by default to the default NPM setting.

Thank you very much for your work!

xavier-hernandez commented 1 year ago

Sorry I really haven't looked into this that deeply. Are you saying for example that if someone tries to hit "try.me.net" and its not set up in NPM its logged?

DerLeole commented 1 year ago

Yes, very much so I think.

I checked the NPM code a bit and the following seems to be the case:

fallback_access.log is just the access log specified in the original nginx.conf files copied over when the docker image is created. I believe this to be the just anything that falls through all other logs. For me it is also filled enmtirely with 444 and 400 responses and nothing else. So likely responses that are malformed go here.

default-host_access.log is used to log everything that arrives correctly at nginx, but has an unknown host (so either just the ip or non set up proxy). Basically whatever happens when the behaviour you set in the Settings Tab of the main menu of NGINX is called. (It is of note that apparently in the logs, if the ip address of the nginx server is directly requested via http, it the host is noted down as "-". If its not too hard, it might be cool to replace that with the server ip before ingesting it into the dashboard).

dead-host-#.log follows the same style as proxy or redirect hosts but is created for any 404 created in NPM.

All in all I think at least the latter two can be safely grouped together, and including fallback into either error or the newly created category for 404/misses could be worth a thought.

xavier-hernandez commented 1 year ago

@Leolele99 So I've updated the develop docker image version and incorporated fallback and dead-host logs into the redirection instance since they have the same log format. The default host has a different log format, maybe I'll create another instance for that but not sure yet.

http://x.x.x.x:7880/redirection/

xavier-hernandez commented 1 year ago

This is now in prod.