elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
30 stars 447 forks source link

[iia]: New Grok for more limited IIS options configured #11756

Open nicpenning opened 6 days ago

nicpenning commented 6 days ago

Integration Name

IIS [iis]

Dataset Name

iis.access

Integration Version

1.20.0

Agent Version

8.15.4

Agent Output Type

elasticsearch

Elasticsearch Version

8.15.4

OS Version and Architecture

Windows Server 2019

Software/API Version

Unknown

Error Message

Provided Grok expressions do not match field value: [2024-11-15 16:25:59 WINYAMS1337 192.168.5.25 GET / helloWorldEatMe 80 yambot500 192.168.5.99 200 0 0 703 7 1]

Event Original

No response

What did you do?

Just tried to use the integration out of the box and none of the IIS logs were parsed for a few servers.

The solution: Created a new GROK expression to catch and IIS servers that use the same limited field configurations to get a match:

%{TIMESTAMP_ISO8601:iis.access.time} (?:-|%{NOTSPACE:iis.access.server_name}) (?:-|%{IPORHOST:destination.address}) (?:-|%{WORD:http.request.method}) (?:-|%{NOTSPACE:_temp_.url_path}) (?:-|%{NOTSPACE:_temp_.url_query}) (?:-|%{NUMBER:destination.port:long}) (?:-|%{NOTSPACE:user.name}) (?:-|%{IPORHOST:source.address}) (%{NUMBER:http.response.status_code:long}) (%{NUMBER:iis.access.sub_status::long}) (?:-|%{NUMBER:iis.access.win32_status:long}) (?:-|%{NUMBER:http.response.body.bytes:long}) (?:-|%{NUMBER:http.request.body.bytes:long}) (?:-|%{NUMBER:_temp_.duration:long})

What did you see?

I saw that this is properly getting parsed in the system ingest pipeline:

{
  "_index": "index",
  "_version": "-3",
  "_id": "id",
  "_source": {
    "iis": {
      "access": {
        "server_name": "WINYAMS1337",
        "sub_status": "0",
        "win32_status": 0
      }
    },
    "@timestamp": "2024-11-15T16:25:59.000Z",
    "ecs": {
      "version": "8.11.0"
    },
    "related": {
      "user": [
        "yambot500"
      ],
      "hosts": [
        "winyams1337"
      ],
      "ip": [
        "192.168.5.99",
        "192.168.5.25"
      ]
    },
    "destination": {
      "address": "192.168.5.25",
      "port": 80,
      "ip": "192.168.5.25"
    },
    "host": {
      "name": "winyams1337"
    },
    "http": {
      "request": {
        "method": "GET",
        "body": {
          "bytes": 7
        }
      },
      "response": {
        "body": {
          "bytes": 703
        },
        "status_code": 200
      }
    },
    "source": {
      "address": "192.168.5.99",
      "ip": "192.168.5.99"
    },
    "event": {
      "duration": 1000000,
      "category": [
        "web",
        "network"
      ],
      "type": [
        "connection"
      ],
      "kind": "event",
      "outcome": "success"
    },
    "user": {
      "name": "yambot500"
    },
    "url": {
      "path": "/",
      "original": "/",
      "query": "helloWorldEatMe"
    }
  },
  "_ingest": {
    "pipeline": "_simulate_pipeline",
    "timestamp": "2024-11-16T04:51:30.4515214Z"
  }
}

What did you expect to see?

I expect that the OOB IIS integration can handle a variety of different configurations for how IIS can be configured. Without this it will be a pain to manage this integration over time since I cannot use this custom GROK with the traditional @custom pipeline.

Anything else?

Nope, pretty straight forward. Just add this new GROK and I will be happy.