vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
18.11k stars 1.6k forks source link

Add new fields to the result of RE parsing #20471

Closed XSWClevo closed 6 months ago

XSWClevo commented 6 months ago

This is my configuration, and the expected effect should be as follows. This is my configuration

[transforms.convert_nginx]
type = "remap"
inputs = ["my_file"]
source = '''
. = parse_syslog(.message) ?? parse_json(.message) ?? parse_glog(.message) ?? parse_klog(.message) ?? parse_nginx_log(.message, "error") ?? parse_common_log(.message) ?? parse_regex_all!(.message, r'^-\s+(?<client>\d{1,3}(?:\.\d{1,3}){3})\s+-\s+\[(?<timestamp>[\S+:]+\s[+\-]\d{4})\]\s+\u0022(?<request>\w+\s+\S+\s+HTTP/1\.\d+)\u0022\s+(?<requestIp>\d{1,3}(?:\.\d{1,3}){3}:\d+)\s+(?<responseStatus>\d+)\s+-\s+(?<size>\d+)\s+\u0022-\u0022\s+\u0022(?<userAgent>\w+\s+\d+.\d+.\d+|\w+\s+\d+.\d+)\u0022\s+-\s+(?<responseTime>\d+.\d+)')

.mw_ip_add = to_string("127.0.0.1")

'''
{
    "client": "10.180.5.183",
    "request": "GET /basic_status HTTP/1.1",
    "requestIp": "10.180.5.183:18080",
    "responseStatus": "200",
    "responseTime": "0.000",
    "size": "112",
    "timestamp": "17/Feb/2024:21:41:57 +0800",
    "userAgent": "Zabbix 6.0.14"
    "my_ip_add": "127.0.0.1"
}

Actual result

{
    "mw_ip_add": "127.0.0.1"
}

The result of putting the configuration in playground.vrl.dev image

XSWClevo commented 6 months ago

Unskilled use of vrl results