Open aensidhe opened 7 years ago
I'm trying to parse nginx access log with additional parameters about upstreams. Here is log record:
[29/Nov/2016:13:01:23 +0000] 172.18.0.1 "GET" "/rest/api/v1/private/explore" "skip=0&take=30" "/rest/api/v1/private/explore?skip=0&take=30" HTTP/1.1 401 983 604 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36" "6-1480424352.352-172.18.0.1-983" "10.0.75.1:8100, 10.0.75.1:8000" "502, 401" "127.275, 0.001" "0, 0" "127.275, 4.085"
And my fluend.conf (it is for development environment, I know it is bad for production):
<source> @type tail path /logs/nginx/access.log read_from_head true <parse> @type grok grok_pattern %{NGINXACCESS} custom_pattern_path /fluentd/etc grok_failure_key grokfailure </parse> tag nginx.access </source> <filter nginx.access> @type mutate replace { "asdsad": "%{upstream_addr}" } split { "upstream_addr": "," } </filter> <match nginx.*> @type stdout </match>
patterns.txt
Output:
2016-11-29 15:54:14.103514500 +0000 nginx.access: {"timestamp":"29/Nov/2016:13:01:23 +0000","clientip":"172.18.0.1","request_method":"\"GET\"","uri":"\"/rest/api/v1/private/explore\"","query":"\"skip=0&take=30\"","request_uri":"\"/rest/api/v1/private/explore?skip=0&take=30\"","server_protocol":"HTTP/1.1","status":"401","request_length":983,"bytes_sent":604,"http_referer":"\"-\"","http_user_agent":"\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36\"","rid":"\"6-1480424352.352-172.18.0.1-983\"","upstream_status":"\"502, 401\"","upstream_connect_time":"\"127.275, 0.001\"","upstream_response_length":"\"0, 0\"","upstream_response_time":"\"127.275, 4.085\"","asdsad":"\"10.0.75.1:8100, 10.0.75.1:8000\""}
Changed output and test to out-stdout.
Ok, got it. prune method of MutateEvent thinks that array is empty. I don't know if it is bug or feature.
prune
MutateEvent
Disclaimer: I do not really know Ruby :)
I'm trying to parse nginx access log with additional parameters about upstreams. Here is log record:
And my fluend.conf (it is for development environment, I know it is bad for production):
patterns.txt
Output: