elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
100 stars 4.92k forks source link

journalbeats syslog facility illegal_argument_exception #12803

Closed hholst80 closed 4 years ago

hholst80 commented 5 years ago

7.2.0 docker container

SYSLOG_FACILITY is assumed to be integer. It is typically not only an integer.

Existing values on my system includes:

$ journalctl SYSLOG_FACILITY=
0           10          4           9           AUDIT       CORE        DHCP4       DNS         PLATFORM    SETTINGS    SUSPEND
1           3           5           AGENTS      BT          DEVICE      DHCP6       MB          RFKILL      SUPPLICANT  WIFI

Example warning:

2019-07-05T19:32:47.400Z    WARN    elasticsearch/client.go:527 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x307b58b0, ext:63692635013, loc:(*time.Location)(0x28b1ee0)}, Meta:common.MapStr(nil), Fields:common.MapStr{"agent":common.MapStr{"ephemeral_id":"27d9da63-de1f-4866-9ad4-d5b80696f3f0", "hostname":"hholst-lt.lan", "id":"cd368456-e42e-4712-b40f-72abd8d547f0", "type":"journalbeat", "version":"7.2.0"}, "ecs":common.MapStr{"version":"1.0.0"}, "event":common.MapStr{"created":common.Time{wall:0x2b760b34, ext:63697951963, loc:(*time.Location)(nil)}}, "host":common.MapStr{"boot_id":"a2d0a5595eb440e7b27386de958402d1", "hostname":"hholst-lt.lan", "id":"bd368c99bfab42b0a3af64000b3f8c1b", "name":"hholst-lt.lan"}, "journald":common.MapStr{"code":common.MapStr{"file":"src/devices/nm-device.c", "line":14710}, "custom":common.MapStr{"nm_device":"wlp4s0", "nm_log_domains":"DEVICE", "nm_log_level":"INFO", "selinux_context":"unconfined\n", "timestamp_boottime":"30532.036327", "timestamp_monotonic":"30527.036327"}}, "message":"<info>  [1557038213.8114] device (wlp4s0): state change: need-auth -> prepare (reason 'none', sys-iface-state: 'managed')", "process":common.MapStr{"capabilites":"200534e2", "cmd":"/usr/sbin/NetworkManager --no-daemon", "executable":"/usr/sbin/NetworkManager", "name":"NetworkManager", "pid":698, "uid":0}, "syslog":common.MapStr{"facility":"DEVICE", "identifier":"NetworkManager", "pid":698, "priority":6}, "systemd":common.MapStr{"cgroup":"/system.slice/NetworkManager.service", "invocation_id":"6583268cfd2046f0aa52ee1adf953584", "slice":"system.slice", "transport":"journal", "unit":"NetworkManager.service"}}, Private:checkpoint.JournalState{Path:"LOCAL_SYSTEM_JOURNAL", Cursor:"s=228c3e521b444c70803e4ba587fee90c;i=6150;b=a2d0a5595eb440e7b27386de958402d1;m=69de0493;t=5881e35a2d48e;x=e771a9f111ceb51d", RealtimeTimestamp:0x5881e35a2d48e, MonotonicTimestamp:0x69de0493}, TimeSeries:false}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [syslog.facility] of type [long] in document with id 'g9yfw2sBvjDm7fG_8E5g'","caused_by":{"type":"illegal_argument_exception","reason":"For input string: \"DEVICE\""}}
h3po commented 5 years ago

I'm also affected in 7.3.0 On my RHEL8 systems, NetworkManager uses facility "DHCP4" for some messages, which causes errors because syslog.facility is defined as type log in fields.yml: https://github.com/elastic/beats/blob/3b20a57693f1cec2937861a541d817354c6c44a5/journalbeat/_meta/fields.common.yml#L297-L298

botelastic[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

botelastic[bot] commented 4 years ago

This issue doesn't have a Team:<team> label.