Closed Joelp closed 3 years ago
jasonish
commented
3 years ago You could try the latest development version: https://evebox.org/files/development/
Sometimes I fix things required due to changes in ElasticSearch, but the supported matrix of Elasticsearch versions, and they speed that they deprecate/release things makes it hard to test all combinations.
Also, what version of Elasticsearch and Logstash? And how are you adding events to Elasticsearch? Just with Logstash?
Joelp
commented
3 years ago Same problem with version 0.13 (latest). I am reading eve.json with logstash and ingesting directly into elasticsearch. This is significant part of logstash config:
input {
file {
path => ["<SOME PATH>/eve.json"]
codec => "json"
type => "eve"
}
}filter {
mutate {
add_tag => "<SOME SERVER CODE NAME>"
rename => { "host" => "host.name" }
}
date {
match => [ "timestamp", "ISO8601" ]
}
if [src_ip] {
geoip {
source => "src_ip"
}
}
if "_geoip_lookup_failure" in [tags] {
mutate {
remove_tag => ["_geoip_lookup_failure"]
}
}
}And finally, this is an example of the log, which EveBox doesn't see and if I open the URL with it, it shows a blank error.
{
"_id": "rm5nR3cBpzK1a1lVvP3e",
"_index": "suricata-alert-prod-2021.01.28",
"_score": 0,
"_source": {
"@timestamp": "2021-01-28T05:11:56.163Z",
"@version": "1",
"alert": {
"action": "blocked",
"category": "Web Application Attack",
"gid": 1,
"metadata": {
"affected_product": [
"Web_Server_Applications"
],
"attack_target": [
"Web_Server"
],
"created_at": [
"2010_07_30"
],
"deployment": [
"Datacenter"
],
"signature_severity": [
"Major"
],
"tag": [
"SQL_Injection"
],
"updated_at": [
"2020_09_14"
]
},
"rev": 5,
"severity": 1,
"signature": "ET WEB_SERVER MYSQL SELECT CONCAT SQL Injection Attempt",
"signature_id": 3220110420
},
"app_proto": "http",
"dest_ip": "<SOME IP>",
"dest_port": 80,
"event_type": "alert",
"flow": {
"bytes_toclient": 66,
"bytes_toserver": 700,
"pkts_toclient": 1,
"pkts_toserver": 3,
"start": "2021-01-28T06:11:54.726904+0100"
},
"flow_id": 1951419592218488,
"geoip": {
"city_name": "Des Moines",
"continent_code": "NA",
"country_code2": "US",
"country_code3": "US",
"country_name": "United States",
"dma_code": 679,
"ip": "<SOME IP>",
"latitude": 41.6005,
"location": {
"lat": 41.6005,
"lon": -93.6091
},
"longitude": -93.6091,
"postal_code": "50307",
"region_code": "IA",
"region_name": "Iowa",
"timezone": "America/Chicago"
},
"host.name": "<SOME HOSTNAME>",
"http": {
"hostname": "<SOME DOMAIN>",
"http_method": "GET",
"http_user_agent": "Mozilla/5.0 (X11; CrOS x86_64 12425.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3877.0 Safari/537.36",
"length": 0,
"protocol": "HTTP/1.1",
"url": "<SOME REFERER>"
},
"in_iface": "<SOME IFACE>",
"path": "<SOME PATH>/eve.json",
"proto": "TCP",
"src_ip": "<SOME SRC IP>",
"src_port": 63606,
"tags": [
"<SOME HOSTNAME>",
"IPv4"
],
"timestamp": "2021-01-28T06:11:56.163345+0100",
"tx_id": 0
},
"_type": "doc"
}
I have to add information, that we are using Suricata 5.0.3 and Elastic 6.8.13.
jasonish
commented
3 years ago I am at a bit of loss right now. I've tested EveBox 0.11.1, 0.12.0 and current development against Elasticsearch/Logstash 6.8.13 and it seems to work fine. I added in your mutate line, and that works as well. Tho I'm surprised it works, as Elastic has dropped support for "." in field names. But maybe thats not enforced in 6, but it is in 7.
One other to try is to specify index index name explicitly to use.. ie) "-i logstash" or whatever on your EveBox command line.
Joelp
commented
3 years ago That's weird.
I have RPM version, so my edit of /etc/sysconfig/evebox is:
# Config file.
CONFIG="-c /etc/evebox/evebox.yaml"
# Other command line options like "-v" for verbose output.
EVEBOX_OPTS="-i suricata-alert-prod-* -v"In OS, I see the process /usr/bin/evebox server -c /etc/evebox/evebox.yaml -i suricata-alert-prod-* -v
But the same problem. This is my journal log:
Jan 29 07:43:55 my-client-server-hostname systemd[1]: Stopping EveBox Server...
Jan 29 07:43:55 my-client-server-hostname systemd[1]: Stopped EveBox Server.
Jan 29 07:43:55 my-client-server-hostname systemd[1]: Started EveBox Server.
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 INFO evebox::version: This is EveBox version 0.12.0 (rev: ba9d586); x86_64-unknown-linux-musl
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 DEBUG evebox::server::main: Certificate checks disabled: true
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 INFO evebox::server::main: Using temporary in-memory configuration database
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 INFO evebox::sqlite::configrepo: Initializing SQLite database
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 INFO evebox::sqlite::configrepo: Updating SQLite database to schema version 1
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 INFO evebox::server::main: Found Elasticsearch version 6.8.13 at http://localhost:9200
Jan 29 07:43:55 my-client-server-hostname evebox[19739]: 2021-01-29 07:43:55 INFO evebox::server::main: Starting server on 127.0.0.1:5636, tls=false
Jan 29 07:47:30 my-client-server-hostname evebox[19739]: 2021-01-29 07:47:30 DEBUG evebox::server::asset: Loading asset index.html
Jan 29 07:47:30 my-client-server-hostname evebox[19739]: 2021-01-29 07:47:30 DEBUG evebox::server::asset: Loading asset styles.ba96b53d8799dc78590f.css
Jan 29 07:47:30 my-client-server-hostname evebox[19739]: 2021-01-29 07:47:30 DEBUG evebox::server::asset: Loading asset runtime.63eafcd730098e79477d.js
Jan 29 07:47:30 my-client-server-hostname evebox[19739]: 2021-01-29 07:47:30 DEBUG evebox::server::asset: Loading asset main.8caba962dcb0463e8b23.js
Jan 29 07:47:30 my-client-server-hostname evebox[19739]: 2021-01-29 07:47:30 DEBUG evebox::server::asset: Loading asset polyfills.08cdd93a9921fbb99ed7.js
Jan 29 07:47:31 my-client-server-hostname evebox[19739]: 2021-01-29 07:47:31 INFO evebox::server::main: Creating anonymous session for user from Some(V4(127.0.0.1:44282)) with name <anonymous>This is my EveBox config:
http:
tls:
enabled: false
database:
type: elasticsearch
elasticsearch:
url: http://localhost:9200
index: suricata-alert-prod-*
disable-certificate-check: true
postgresql:
managed: true
authentication:
required: no
type: usernamepassword
input:
enabled: false
custom-fields:
geoip:
disabled: false
database: /etc/evebox/GeoLite2-City_20201013/GeoLite2-City.mmdb
event-services:
- type: custom
enabled: false
name: Scirius
event-types:
- alert
url: https://10.16.1.179/rules/rule/{{alert.signature_id}}
- type: custom
enabled: false
name: Dumpy
url: http://10.16.1.1:7000/?event={{raw}}
target: newI read issue #149 and I think that my template might be good to show:
...
"mappings" : {
"doc" : {
"properties" : {
"icmp_type" : {
"type" : "long"
},
"program" : {
"type" : "keyword"
},
"src_ip" : {
"type" : "ip"
},
"event_type" : {
"type" : "keyword"
},
"alert" : {
"dynamic" : true,
"properties" : {
"severity" : {
"type" : "long"
},
"rev" : {
"type" : "long"
},
"signature_id" : {
"type" : "long"
},
"gid" : {
"type" : "long"
},
"signature" : {
"type" : "keyword"
},
"action" : {
"type" : "keyword"
},
"category" : {
"type" : "keyword"
}
}
},
"flow_id" : {
"type" : "keyword"
},
"beat" : {
"dynamic" : true,
"properties" : {
"hostname" : {
"type" : "keyword"
}
}
},
"icmp_code" : {
"type" : "long"
},
"host.name" : {
"type" : "keyword"
},
"id" : {
"type" : "integer"
},
"flow" : {
"dynamic" : true,
"properties" : {
"reason" : {
"type" : "keyword"
},
"pkts_toserver" : {
"type" : "long"
},
"alerted" : {
"type" : "boolean"
},
"start" : {
"type" : "date"
},
"bytes_toclient" : {
"type" : "long"
},
"end" : {
"type" : "date"
},
"state" : {
"type" : "keyword"
},
"bytes_toserver" : {
"type" : "long"
},
"pkts_toclient" : {
"type" : "long"
},
"age" : {
"type" : "long"
}
}
},
"dest_port" : {
"type" : "long"
},
"timestamp" : {
"type" : "date"
},
"geoip" : {
"dynamic" : true,
"properties" : {
"timezone" : {
"type" : "keyword"
},
"ip" : {
"type" : "ip"
},
"latitude" : {
"type" : "half_float"
},
"continent_code" : {
"type" : "keyword"
},
"city_name" : {
"type" : "keyword"
},
"country_code2" : {
"type" : "keyword"
},
"country_name" : {
"type" : "keyword"
},
"country_code3" : {
"type" : "keyword"
},
"location" : {
"type" : "geo_point"
},
"region_name" : {
"type" : "keyword"
},
"postal_code" : {
"type" : "keyword"
},
"longitude" : {
"type" : "half_float"
},
"region_code" : {
"type" : "keyword"
}
}
},
"app_proto_tc" : {
"type" : "keyword"
},
"message" : {
"type" : "keyword"
},
"tx_id" : {
"type" : "long"
},
"app_proto" : {
"type" : "keyword"
},
"tags" : {
"type" : "keyword"
},
"src_port" : {
"type" : "long"
},
"in_iface" : {
"type" : "keyword"
},
"proto" : {
"type" : "keyword"
},
"dest_ip" : {
"type" : "ip"
},
"http" : {
"dynamic" : true,
"properties" : {
"redirect" : {
"type" : "keyword"
},
"protocol" : {
"type" : "keyword"
},
"hostname" : {
"type" : "keyword"
},
"http_method" : {
"type" : "keyword"
},
"http_content_type" : {
"type" : "keyword"
},
"http_refer" : {
"type" : "keyword"
},
"length" : {
"type" : "long"
},
"url" : {
"type" : "keyword"
},
"status" : {
"type" : "long"
},
"http_user_agent" : {
"type" : "keyword"
}
}
},
"app_proto_ts" : {
"type" : "keyword"
},
"fields" : {
"dynamic" : true,
"properties" : {
"event_category" : {
"type" : "keyword"
}
}
}
}
}
},
...Finally, I had a little time to play with. I upgraded Elastic to 7.10.2 with security enabled. There are all logs in EveBox 0.11.1. But after upgrade to 0.12.0, there is No events found. message. And journal log is full of messages:
Mar 15 13:15:10 elastic-node04-cn evebox[600627]: 2021-03-15 13:15:10 ERROR evebox::eve::processor: Failed to read event: io error: No such file or directory (os error 2)
I tried version 0.13.0-0.dev1614492101 and have No events found. too. But without any journal log like in v0.12.
And finaly, I tried index logstash-* with default template and there is the same problem.
Joelp
commented
3 years ago I found the problem. EveBox has some hard coded index name inside. As you can see above, my EveBox config has line index: suricata-alert-prod-*. This work's in EveBox 0.11. But after testing upgrade to 0.13, I saw this:
2021-03-22 12:53:25 DEBUG evebox::server::main: Elasticsearch base index: suricata-alert-prod-*
2021-03-22 12:53:25 DEBUG evebox::server::main: Elasticsearch search index pattern: suricata-alert-prod-*-*
That's wrong. So, I changed line to index: suricata-alert-prod, and EveBox works again:
2021-03-22 12:55:38 DEBUG evebox::server::main: Elasticsearch base index: suricata-alert-prod
2021-03-22 12:55:38 DEBUG evebox::server::main: Elasticsearch search index pattern: suricata-alert-prod-*I'm not sure that this is good behaviour. What if I want to use specific index name? I think that better is to use what is exactly in config file.
jasonish
commented
3 years ago Ahh.. Makes sense. The "index" is an index prefix, I see I don't refer to it as an index prefix. Looks like I dropped some code that removed the "-" if the user provided it. Anyways expected behaviour is to use an index parameter like "logstash", EveBox will then search "logstash-". If you need to use the index name exactly as provided, as of 0.13.0 you can add --no-index-suffix which will use the index name as provided.
jasonish
commented
3 years ago EVEBOX_OPTS="-i suricata-alert-prod-* -v"
Sorry, I should have caught this here...
jasonish
commented
3 years ago Closing as I think a reasonable work-around exists. Please re-open if you think this case could be handled better.
When I update EveBox from version 0.11.1 to 0.12.0, a lot of events disappear. When I downgrade back to 0.11.1, I see all events. Unfortunately, there is silence about any issue in the journal log.
EveBox 0.11.1
EveBox 0.12.0
Only mutate, in logstash, what I do is:
rename => { "host" => "host.name" }This is because of compatibility with filebeat. But I tried a new index without this mutation and there is the same problem.
Do you have any clue how to investigate what the problem is with that?