search

magnusbaeck / logstash-filter-verifier

Apache License 2.0
192 stars 27 forks source link

Typo fix #22

Closed matejzero closed 7 years ago

matejzero commented 7 years ago

I think this should be fixed as part of 90e9e4b as well.

matejzero commented 7 years ago

If I run with testcases in my config file eg.

{
  "fields": {
    "appid": "nagios",
    "type": "icinga"
  },
  "ignore": [
    "logstash_node"
  ],
  "testcases": {
    "input": [
      "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%"
    ],
    "expected": {
      "@timestamp": "2017-05-20T10:20:30.000Z",
      "host": "hostX",
      "appid": "nagios",
      "type": "icinga-hw",
      "message": "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%",
      "check_type": "HOST ALERT",
      "state": "DOWN",
      "statelevel": "SOFT",
      "attempt": "1",
      "check_message": "CRITICAL - 10.20.30.40: rta nan, lost 100%",
      "grafana_desc": "Host ALERT - hostX - DOWN",
      "grafana_msg": "State: SOFT | Reason: CRITICAL - 10.20.30.40: rta nan, lost 100%.",
      "tags": [ "_grok_nagios_success"]
    }
  }
}

I get an error: Error reading/unmarshalling ./tests/nagios.json: json: cannot unmarshal object into Go value of type []testcase.TestCase

However, it works when using testcaseset.

Or maybe I'm using it the wrong way, but that way made sense to me reading the README.

magnusbaeck commented 7 years ago

Aha! Your expected value needs to be a list of objects:

"expected": [
  {
    "@timestamp": "2017-05-20T10:20:30.000Z",
    "host": "hostX",
    "appid": "nagios",
    "type": "icinga-hw",
    "message": "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%",
    "check_type": "HOST ALERT",
    "state": "DOWN",
    "statelevel": "SOFT",
    "attempt": "1",
    "check_message": "CRITICAL - 10.20.30.40: rta nan, lost 100%",
    "grafana_desc": "Host ALERT - hostX - DOWN",
    "grafana_msg": "State: SOFT | Reason: CRITICAL - 10.20.30.40: rta nan, lost 100%.",
    "tags": [ "_grok_nagios_success"]
  }
]
matejzero commented 7 years ago

Still the same error.

{
  "fields": {
    "appid": "nagios",
    "type": "icinga"
  },
  "ignore": [
    "logstash_node"
  ],
  "testcases": {
    "input": [
      "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%"
    ],
    "expected": [
      {
        "@timestamp": "2017-05-20T10:20:30.000Z",
        "host": "hostX",
        "appid": "nagios",
        "type": "icinga-hw",
        "message": "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%",
        "check_type": "HOST ALERT",
        "state": "DOWN",
        "statelevel": "SOFT",
        "attempt": "1",
        "check_message": "CRITICAL - 10.20.30.40: rta nan, lost 100%",
        "grafana_desc": "Host ALERT - hostX - DOWN",
        "grafana_msg": "State: SOFT | Reason: CRITICAL - 10.20.30.40: rta nan, lost 100%.",
        "tags": [ "_grok_nagios_success"]
      }
    ]
  }
}
matejzero commented 7 years ago

Nevermind. I used breml's oneliner to convert tests posted in #23.

Maybe we could add an example to README.

{
  "fields": {
    "appid": "nagios",
    "type": "icinga-hw"
  },
  "ignore": [
    "logstash_node"
  ],
  "testcases": [
    {
      "input": [
        "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%"
      ],
      "expected": [
        {
          "@timestamp": "2017-05-20T10:20:30.000Z",
          "host": "hostX",
          "appid": "nagios",
          "type": "icinga-hw",
          "message": "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%",
          "check_type": "HOST ALERT",
          "state": "DOWN",
          "statelevel": "SOFT",
          "attempt": "1",
          "check_message": "CRITICAL - 10.20.30.40: rta nan, lost 100%",
          "grafana_desc": "Host ALERT - hostX - DOWN",
          "grafana_msg": "State: SOFT | Reason: CRITICAL - 10.20.30.40: rta nan, lost 100%.",
          "tags": [ "_grok_syslog_prefilter_skipped", "_grok_nagios_success", "save_to_elasticsearch" ]
        }
      ]
    }
  ]
}
magnusbaeck commented 7 years ago

The testcases value itself needs to be a list. Here's a complete example:

{
    "fields": {
        "type": "http-access"
    },
    "ignore": ["host"],
    "testcases": [
        {
            "input": [
                "127.0.0.1 - - [28/Jul/2016:11:02:14 +0200] \"GET /foo/bar?query HTTP/1.1\" 200 580 \"http://www.example.com/fnord\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36\""
            ],
            "expected": [
                {
                    "@timestamp": "2016-07-28T09:02:14.000Z",
                    "client": {
                        "ip": "127.0.0.1",
                        "name": "localhost"
                    },
                    "http_ident": "-",
                    "http_method": "GET",
                    "http_referer": "http://www.example.com/fnord",
                    "http_request": "/foo/bar?query",
                    "http_response": 200,
                    "http_version": "1.1",
                    "response_size": 580,
                    "type": "http-access",
                    "user": "-",
                    "useragent": {
                        "device": "Other",
                        "major": "52",
                        "minor": "0",
                        "name": "Chrome",
                        "os": "Linux",
                        "os_name": "Linux",
                        "patch": "2743",
                        "raw": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36"
                    }
                }
            ]
        },
        {
            "input": [
                "www.example.com:80 127.0.0.1 - - [28/Jul/2016:11:02:14 +0200] \"GET /foo/bar?query HTTP/1.1\" 200 580 \"http://www.example.com/fnord\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36\""
            ],
            "expected": [
                {
                    "@timestamp": "2016-07-28T09:02:14.000Z",
                    "client": {
                        "ip": "127.0.0.1",
                        "name": "localhost"
                    },
                    "http_ident": "-",
                    "http_method": "GET",
                    "http_referer": "http://www.example.com/fnord",
                    "http_request": "/foo/bar?query",
                    "http_response": 200,
                    "http_version": "1.1",
                    "port": 80,
                    "response_size": 580,
                    "type": "http-access",
                    "user": "-",
                    "useragent": {
                        "device": "Other",
                        "major": "52",
                        "minor": "0",
                        "name": "Chrome",
                        "os": "Linux",
                        "os_name": "Linux",
                        "patch": "2743",
                        "raw": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Safari/537.36"
                    },
                    "virtualhost": "www.example.com"
                }
            ]
        }
    ]
}
matejzero commented 7 years ago

You beat me to it:)

Ok, this was only my misunderstanding, so I'm closing this.

breml commented 7 years ago

@matejzero The problem in your config is, that testcases needs to be an array and not an hash. So the correct version of your config would be:

{
  "fields": {
    "appid": "nagios",
    "type": "icinga"
  },
  "ignore": [
    "logstash_node"
  ],
  "testcases": [ {
    "input": [
      "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%"
    ],
    "expected": [
      {
        "@timestamp": "2017-05-20T10:20:30.000Z",
        "host": "hostX",
        "appid": "nagios",
        "type": "icinga-hw",
        "message": "[1495275630] HOST ALERT: hostX;DOWN;SOFT;1;CRITICAL - 10.20.30.40: rta nan, lost 100%",
        "check_type": "HOST ALERT",
        "state": "DOWN",
        "statelevel": "SOFT",
        "attempt": "1",
        "check_message": "CRITICAL - 10.20.30.40: rta nan, lost 100%",
        "grafana_desc": "Host ALERT - hostX - DOWN",
        "grafana_msg": "State: SOFT | Reason: CRITICAL - 10.20.30.40: rta nan, lost 100%.",
        "tags": [ "_grok_nagios_success"]
      }
    ]
  } ]
}

The important parts are testcases: [ { and the } ] on the second last line.