urtho
commented
6 months ago In Coolify v4 you can enable access logging by:
- adding a log drain
- adding this to proxy config on each server and restarting proxy (does not work on swarm as of 4.0.0-beta.172)
logging:
driver: fluentd
options:
fluentd-address: 'tcp://127.0.0.1:24224'
fluentd-async: 'true'
fluentd-sub-second-precision: 'true'
command:
- '--accesslog=true'
- '--accesslog.format=json'
- '--accesslog.fields.defaultmode=drop'
- '--accesslog.fields.names.ClientHost=keep'
- '--accesslog.fields.names.DownstreamContentSize=keep'
- '--accesslog.fields.names.DownstreamStatus=keep'
- '--accesslog.fields.names.Duration=keep'
- '--accesslog.fields.names.RequestHost=keep'
- '--accesslog.fields.names.RequestMethod=keep'
- '--accesslog.fields.names.RequestPath=keep'
- '--accesslog.fields.names.RequestReferer=keep'
- '--accesslog.fields.headers.defaultmode=drop'
- '--accesslog.fields.headers.names.user-agent=keep'
- '--accesslog.fields.headers.names.referer=keep'
- '--accesslog.fields.headers.names.cf-ray=keep'
- '--accesslog.fields.headers.names.cf-ipcountry=keep'
- '--accesslog.fields.headers.names.cf-connecting-ip=keep'If you fancy Graylog you can forward logs by setting custom fluentbit config to :
[INPUT]
Name forward
Tag cool-stg
Listen 0.0.0.0
Port 24224
Buffer_Chunk_Size 32KB
Buffer_Max_Size 64KB
[FILTER]
Name record_modifier
Match *
Record hostname cadm-stg
Remove_key container_id
Remove_key source
Remove_key function
Remove_key file
Remove_key msg
Remove_key line
[OUTPUT]
Name gelf
Match *
Host YOURGRAYLOGIP
Port 12201
Mode udp
Gelf_Short_Message_Key log
AlejandroAkbal
Is there an existing issue for this?
Summary
Coolify should expose access logs created by Traefik on the host to enable analyzation of requests across applications. Since log analyzers are often times not coolify-able, the logs should either be accessible via syslog or (preferably) in a dedicated volume on the host, where they're rotated automatically.
Why should this be worked on?
Security
Tools like Fail2Ban (first released 2004) and the more modern, crowdsourced IPS Crowdsec are parsing and analyzing logs from a variety of applications in many formats to monitor security. Based on events logged, they are able to block IP Addresses or Address-Spaces and are able to distribute this data to others. Since the traefik proxy currently deployed by coolify does not have access logging enabled, this log analyzation is not possible. Exposing access logs on the host would enable users to keep using their tools, without compromising on the security monitoring they already built.
Analytics
Users might want to track the requests to their infrastructure with a single tool. GoAccess for example can analyze web logs from a variety of formats and break down visitors into User Agent, sites visited, referrer, etc. This of course is only possible with the web logs from all the virtual hosts accessible. By exposing access logs on the host, coolify can give its users the ability to analyze their traffic without having to implement an analytics tool itself.
Centralized Log Management
Users might want to feed all application/access logs into one management tool like Splunk or DataDog. This enables recognizing issues with applications early on across all of their infrastructure. This is not possible without exposing traefik's access logs on the host machine. Implementing external logging into every single application can be cumbersome, and in many cases, impossible.
Fider: https://feedback.coolify.io/posts/195/traefik-access-logs-on-host