Closed pmhausen closed 6 months ago
Yeah the problem is the syslog-ng integration, it requires a very specific format so that the default parser and log view in the opnsense can display it.
/usr/local/etc/syslog-ng.conf.d/caddy.conf
The caddy logs in standard json format are sent to the unix datagram socket /var/caddy/var/run/log
and from there are caught by syslog-ng. In the above configuration are specific parsing rules that rewrite the logs. Then it's written into log files.
I have an older branch that doesn't have the syslog-ng integration yet, the Caddyfile template would have to be changed like this in the global options:
log {
output file /var/log/caddy/caddy.log {
roll_size 10MiB
roll_keep 10
roll_keep_for 720h
}
}
Though doing that comes with the regression that the log view in the OPNsense wouldn't work properly anymore.
You would need to use this link then /ui/diagnostics/log/caddy/caddy
. You can see that the unformatted json is displayed in the GUI.
Ah ... understood. May I suggest a new feature? ;)
Might even send a pull request later this evening, I'll look into it. Unless you like the idea and you are faster,
Kind regards, Patrick
P.S. Initial motivation was that I had CrowdSec in a distributed setup with a log collector on my standalone Caddy and engine and bounce on OPNsense, but both the OPNsense plugin as well as the FreeBSD package notoriously overwrite the local API settings with "localhost" instead of a remote engine, so I was always busy fixing the setup. Looks like it's best run on a single system.
This requires some more logic in the Caddyfile template. Right now, the http access logs are added via "include" with the UUID of said domain. That in return makes these logs appear in the syslog-ng logs.
Maybe a global option could be added, that would be like "Log HTTP Access Logs to seperate Files", or something like that. It would then write actual "log option blocks" with the UUID.log of the domain as log files into each Domain that has the "Http Access Logs" checkbox enabled, instead of creating the include statement in the main log.
Example:
example.com {
log {
output file /var/log/access_UUID.log {
roll_size 10mb
roll_keep 5
roll_keep_for 720h
}
}
This can be a little tricky, the template is already the hardest part (for me) to maintain. But I think its possible without making it too convoluted. Also, it's important to work with UUIDs in the log names, since only domain names are a little dangerous since Caddy hates duplicate names.
Good idea. The template is not that hard, actually. You can loop in Jinja.
Using the access uuid for the filename is also a great idea, that avoids problems with wildcard domains etc. And the logs are parsed automatically, anyway, and if you must look, there's grep.
I'll look into it. Isn't Open Source wonderful? :)
Sure you can try, if you're stuck I can give it a shot too. ^^
Done - thanks.
I am moving my reverse proxy from a standalone installation to your plugin specifically to have CrowdSec parse the access logs and block malicious requests.
For some reason the log files contain invalid json. Each line starts like this:
Any idea what is going wrong? On my old standalone system logs look like this:
Kind regards, Patrick