search

dmikusa / cf-php-apache-buildpack

CloudFoundry PHP & Apache HTTPD Buildpack
Apache License 2.0
4 stars 11 forks source link

Add support for outputing apache access and error logs to loggregator #8

Open gberche-orange opened 10 years ago

gberche-orange commented 10 years ago

The CloudFoundry recommended way to collect HTTP access logs is to have apps dump them on stdout, in order for the loggregator to collect them.

See https://docs.google.com/document/d/1zTbS9f6s1Y8WYMhJv5lu6G1bf1ktXA-y-itc3ZOhVAA/edit?disco=AAAAAGhrT_w# for more details.

Current httd-logging.conf is:

ErrorLog "logs/error_log"
LogLevel warn
<IfModule log_config_module>
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common
    <IfModule logio_module>
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>
    CustomLog "logs/access_log" common
</IfModule>

It would be great to have the cf-php-apache-buildpack configure apache to dump access logs to stdout, with a prefix added on each line in order to distinguish these logs from app-generated logs.

Suggesting to add the prefix "s=access" for a concise short-cut for source=apache_access_logs

In terms of implementation, it is not clear whether standard apache config is able to generate logs on stdout, see http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#customlog may be though piped streams ?

This php heroku buildpack was instead tailing the access file on disk, see http://labs.enru.co.uk/phponheroku/#17 However, this might lack some additional logrotate to avoid filling up the disk ?

The same holds for error logs which could be output to stderr with the "s=apache_err" prefix

dmikusa commented 10 years ago

This is definitely something that I want to add. I'm in the middle of restructuring the build pack at the moment. After that work is done, I'll make sure to add this.

In regards to prefix, I'm not exactly sure I understand what you mean there. Do you mean the LogFormat should be prepended with the literal text "s=access"? Is this something understood by loggregator? or is it purely for organizational purposes?

LogFormat "s=access %h %l %u %t \"%r\" %>s %b" common

In regards to the ErrorLog, I think you can format this, but only in 2.4.x. I don't think 2.2.x supports custom formats.

gberche-orange commented 10 years ago

Great thanks!

The prefix won't be processed by the loggregator but is a way to add meta data to the log entries to that clients can sort them and try to reconstitute distinct files out of the stdout/stderr streams. Having the prefix formatted as a key/value pair makes it easy for most log processors to then filter logs using them.