Unable to parse logs with common log format

[vinbh]

Hi Martin,

when using the log format as common in my nginx log lines, they are not being parsed and I get this error.

error while parsing line '1.1.1.1 - - [05/Oct/2023:00:50:04 +0000] "GET /xxx/xxx.pkg HTTP/1.1" 400 224 "-" "pkg/1.17.2"': text log parsing err: access log line '3.18.46.197 - - [05/Oct/2023:00:50:04 +0000] "GET /xxx/xxx/xxxxx/xxx.pkg HTTP/1.1" 400 224 "-" "pkg/1.17.2"' does not match given format '^common'

Steps to reproduce the behavior (as minimally and precisely as possible) Even when running with systemd using config file below, it fails to parse logs.

$ ./prometheus-nginxlog-exporter \ -format="common" \ -listen-port=4040 \ -namespace=nginx \ [PATHS-TO-LOGFILES...]

Expected behavior Expect to see metrics at ip:4040/metrics

Configuration file (remove section, if not applicable):

listen {
  port = 4040
}

namespace "nginx" {
  source = {
    files = [
      "/home/admin/nginx_logs/access.log"
    ]
  }
  format = "$remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$http_x_forwarded_for\""

  labels {
    app = "default"
  }
}

Example log file (remove section, if not applicable):

1.1.1.1 - - [05/Oct/2023:00:00:08 +0000] "GET /xx/xx/xxx/meta.conf HTTP/1.1" 400 224 "-" "pkg/1.17.2"
1.1.1.1 - - [05/Oct/2023:00:00:08 +0000] "GET /xx/xx/xxx//meta.txz HTTP/1.1" 400 224 "-" "pkg/1.17.2"
1.1.1.1 - - [05/Oct/2023:00:00:08 +0000] "GET /xx/xx/xxx/meta.txz HTTP/1.1" 400 224 "-" "pkg/1.17.2"

Metrics output (remove section, if not applicable):

error while parsing line '1.1.1.1 - - [05/Oct/2023:00:50:04 +0000] "GET /xxx/xxx.pkg HTTP/1.1" 400 224 "-" "pkg/1.17.2"': text log parsing err: access log line '1.1.1.1 - - [05/Oct/2023:00:50:04 +0000] "GET /xxx/xxx/xxxxx/xxx.pkg HTTP/1.1" 400 224 "-" "pkg/1.17.2"' does not match given format '^common'

Environment:

• Exporter version:
• OS Debian 11
• Deployment method : debain install, running using systemd, and binary. (not together.)

[martin-helmich]

Hey,

your -format command-line flag overrides your format value in your configuration file, and the usual aliases for common log formats (like common) are not supported; regardless if on the command-line or configuration file, you'll need to specify the full log format.

Hope that helps!

[helmich-bot]

There has not been any activity to this issue in the last 30 days. It will automatically be closed after 7 more days. Remove the stale label to prevent this.