[Support] Evebox events but not alerts

[Veshialle]

Hello, We are deploying suricata using rocknsm which use filebeat to send json from eve.json to kafka, which finally send it to logstash and then elasticsearch (yeah it's complicating our life). Each nids is remote and send data to a centralized cluster of elastic stack. So we can't install evebox in each nids. The logstash pipeline from rocknsm is way huge, with tons of enrichment and converting most of the fields in ecs. I tried to maintain all of the fields originally in the eve.json so evebox should read all of them correctly.

In fact we have all the events like:

And filtering them for alerts we got:

But checking in the inbox tab (or alerts tab) we got:

We're also trying to double the output from logstash, with the raw packets from eve.json everything is fine (but we prefer to not use this "solution" because of doubled internet traffic and doubled space inside our cluster).

I can post the duplicated json(s) from an example alert, to not making this unreadable I will share a pastebin: Raw data directly from eve.json and sent to elasticsearch via logstash.

Big data enriched with our pipeline

And here the database configuration inside evebox.yaml

# Database configuration.
database:

  # Database type: elasticsearch, sqlite.
  type: elasticsearch

  elasticsearch:
    url: http://127.0.0.1:9200
    # production-style index
    index: ecs-suricata2-network
    # demo-style index
    # index: test-suricata
    disable-certificate-check: true

    # The keyword to use for terms query. EveBox will do its best to
    # figure this out on its own, but if you need to override it, you
    # can do so here. The usual values are:
    #    raw     -> Logstash / Elastic Search < 5.
    #    keyword -> Logstash / Elastic Search >= 5.
    #    ""      -> Filebeat / Elastic Search >= 5.
    # Note that a quoted empty string is required to force an empty string.
    #keyword: ""

    username: itsnottherealusername
    password: itsnottherealpassword

    # For loading the EveBox template (Logstash compatible) into
    # Elasticsearch. It is recommended to turn this option on if only
    # using EveBox to add events to Elasticsearch. Leave disabled if
    # already using Logstash or Filebeat on the same index.
    #
    # Default: false
    #force-template: false

Which could be the issue in the configuration?

[jasonish]

I think somewhere in your transformations, the event is being modified in such a way that my aggregation queries to build the inbox are no longer working, and Elastic is pretty strict in terms of schema.

EveBox currently works with:

• the default logstash template for adding events
• the default filebeat (without the suricata module) for adding events

Unforunately, as soon as anything deviates from this template/mappings, things don't work very well.

I'm going to start adding support for ECS real soon. This will handle events that are added with the Filebeat Suricata module which does an ECS transformation on them. Part of this work will likely require coming up with some custom mapping configurations, which may allow some support for mappings that don't align strictly with the default Logstash, Filebeat or ECS layouts. That might help you.

What might be useful is capting the Elasticsearch log when the Inbox page is loaded. This might tell us why its failing to build the result set.

[Veshialle]

Thank you for your reply. I've seen the migration you're working on migrating evebox in ecs. It would be really worth that but It's better for us to get what is not working right now.

Does queries and aggregations are inside elastic.js? In this case We can test queries and find what is the problems with our transformations. More or less i've already checked that all the fields inside the suricata raw json are in our document.

Which fields EveBox expect to find? At least where can I check this in the source code? It would be great.

Thank you very much.

[jasonish]

Here's an inbox query... elastic.js is a bit out of date, and was more for some reference while porting from Go to Rust.. Here is a full Inbox query:

https://gist.github.com/jasonish/db9e6d633d2174816f7738237b263e5a

I was able to use this from the Kibana dev tools to get results, but should be adaptable to curl or Postman.

Even if the field names are the same and in the right place, a custom mapping could cause the search to fail. You should see that in the error returned though.

[jasonish]

Or as Curl:

curl -XPOST "http://elasticsearch:9200/logstash-*/_search" -H 'Content-Type: application/json' -d'{  "aggs": {    "signatures": {      "aggs": {        "sources": {          "aggs": {            "destinations": {              "aggs": {                "escalated": {                  "filter": {                    "term": {                      "tags": "evebox.escalated"                    }                  }                },                "newest": {                  "top_hits": {                    "size": 1,                    "sort": [                      {                        "@timestamp": {                          "order": "desc"                        }                      }                    ]                  }                },                "oldest": {                  "top_hits": {                    "size": 1,                    "sort": [                      {                        "@timestamp": {                          "order": "asc"                        }                      }                    ]                  }                }              },              "terms": {                "field": "dest_ip.keyword",                "size": 10000              }            }          },          "terms": {            "field": "src_ip.keyword",            "size": 10000          }        }      },      "terms": {        "field": "alert.signature_id",        "size": 10000      }    }  },  "query": {    "bool": {      "filter": [        {          "exists": {            "field": "event_type"          }        },        {          "term": {            "event_type": "alert"          }        },        {          "range": {            "@timestamp": {              "gte": "2020-09-24T17:55:29.108Z"            }          }        }      ],      "must_not": [        {          "term": {            "tags": "evebox.archived"          }        }      ]    }  },  "sort": [    {      "@timestamp": {        "order": "desc"      }    }  ]}'

[Veshialle]

Sorry for my (very) late reply but I had some trouble getting up and running the IDS.

I found some fields that are mapped in different way from my template and clean teamplate. All of them fixed and now it's working everything.

Can I ask if you have a roadmap on how and when implement the ecs in evebox?

Thank you very much!

[jasonish]

Can I ask if you have a roadmap on how and when implement the ecs in evebox?

I am about to release 0.12.0. There is not much visible in this release to end users other than some Elasticsearch fixups, and me getting myself out of some holes I coded myself into.

ECS is top priority for what will be 0.13.0, followed by adding some reporting functionality with the SQLite backend.

[Veshialle]

Sorry if I keep this issue opened, if You wish I can open another one about another question: How is the process of escalate / de-escalate and archive? I mean, from queries you showed the process will tag the document from elasticsearch. Does this really update the document on elastic? Because trying to escalate an alert this does not "appear" in escalated tab.

Thank you very much!

[jasonish]

Escalate runs script on all matching documents to add the tag. A request to escalate looks something like this:

POST logstash-*/_update_by_query?refresh=true&conflicts=proceed

{
  "query": {
    "bool": {
      "filter": [
        {
          "exists": {
            "field": "event_type"
          }
        },
        {
          "term": {
            "event_type.keyword": "alert"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2020-10-07T21:57:48.327Z",
              "lte": "2020-10-07T21:57:48.327Z"
            }
          }
        },
        {
          "term": {
            "src_ip.keyword": "10.16.3.0"
          }
        },
        {
          "term": {
            "dest_ip.keyword": "44.239.243.142"
          }
        },
        {
          "term": {
            "alert.signature_id": 906200063
          }
        }
      ],
      "must_not": [
        {
          "term": {
            "tags": "evebox.escalated"
          }
        }
      ]
    }
  },
  "script": {
    "inline": "\n                if (params.tags != null) {\n                    if (ctx._source.tags == null) {\n                        ctx._source.tags = new ArrayList();\n                    }\n                    for (tag in params.tags) {\n                        if (!ctx._source.tags.contains(tag)) {\n                            ctx._source.tags.add(tag);\n                        }\n                    }\n                }\n                if (ctx._source.evebox == null) {\n                    ctx._source.evebox = new HashMap();\n                }\n                if (ctx._source.evebox.history == null) {\n                    ctx._source.evebox.history = new ArrayList();\n                }\n                ctx._source.evebox.history.add(params.action);\n            ",
    "lang": "painless",
    "params": {
      "action": {
        "action": "escalated",
        "timestamp": "2020-10-07T22:08:20.577Z",
        "username": "<anonymous>"
      },
      "tags": [
        "evebox.escalated"
      ]
    }
  }
}

[Veshialle]

Hi, Sorry for leaving this issue opened. I've updated version of evebox (via yum; installed version 0.12.0). Using standard pipeline for logstash (as you mentioned above, default template for json(s)) I'm having some trouble navigating alerts: In facts if I select a single alerts from the Alert page I got this:

[jasonish]

Closing. Please try with latest release if still using EveBox and create a new issue as needed.