Question about fields

[cwiechmann]

I have the following Filebeat config, which is setting the type based on the input Log-File:

- input_type: log
  paths: ["/var/log/work/*.log"]
  fields:
    type: openlog
- input_type: log
  paths: ["/var/log/trace/*.trc"]
  fields:
    type: trace

In my logstash config, when using the following if clause, it works as expected:

if ([fields][type] == "openlog") {

This behavior I would like to test with the LFV, hence I configured my test like so:

{
    "fields": {
      "type": "openlog"
    },
    "codec": "json_lines",
    "testcases": [
      {

but this doesn't work. It only works, when changing my logstash config to this expression: if ([type] == "openlog") {, but then it is no longer working with Filebeat.

Do I missed anything?

[cwiechmann]

I have discovered an option in filebeat to place configured fields under root. With that set to true, the LFV behaves as Filebeat. Perhaps a note in the readme might be useful.

- input_type: log
  paths: ["/var/log/work/*.log"]
  fields_under_root: true
  fields:
    type: openlog
- input_type: log
  paths: ["/var/log/trace/*.trc"]
  fields_under_root: true
  fields:
    type: trace

[magnusbaeck]

I find the Beats's default behavior of placing custom fields as subfields of fields rather silly, but LFV supports it via

{
  "fields": {
    "fields": {
      "type": "openlog"
    }
  },

or slightly shorter (with LFV 1.6+):

{
  "fields": {
    "[fields][type]": "openlog"
  },

It's not clear to me what an addition to the readme file should look like. Explaining it with sufficient context probably requires at least one paragraph of text, and that seems like a lot for something that's really a quirk in one of many possible event sources (albeit one of the most common ones).

[cwiechmann]

Thanks @magnusbaeck for the feedback. I agree, that the default behavior of Beats is not best to have such custom fields hard-coded into fields. I created a PR to add a special section for Beats underneath Syslog as I was thinking it makes sense at this place.