search

jasonish / evebox

Web Based Event Viewer (GUI) for Suricata EVE Events in Elastic Search
https://evebox.org/
MIT License
432 stars 67 forks source link

EveBox doesn't show alerts in the browser #145

Closed fluxcap1 closed 4 years ago

fluxcap1 commented 4 years ago

I can't see any data in the browser.

I've compiled EveBox 0.11.1 on openSUSE. EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64

System -

Linux flux 5.3.18-lp152.36-default #1 SMP Tue Aug 18 17:09:44 UTC 2020 (885251f) x86_64 x86_64 x86_64 GNU/Linux

I start evebox as - /usr/local/evebox/evebox -c /usr/local/evebox/evebox.yaml -v

Output -

`[Wed Aug 26 22:19:03 root@flux /usr/local/evebox]

/usr/local/evebox/evebox server -c /usr/local/evebox/evebox.yaml -v

2020-08-26 22:19:19 (server.go:163) -- This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64 2020-08-26 22:19:19 (server.go:252) -- Self test: found embedded index.html. 2020-08-26 22:19:19 (geoip-service.go:44) -- Failed to initialize geoip database: no database files found 2020-08-26 22:19:19 (configdb.go:59) -- Using configuration database file /var/lib/evebox/config.sqlite 2020-08-26 22:19:19 (migrator.go:66) -- Current database schema version: 1 2020-08-26 22:19:19 (sqlite.go:140) -- Configuring SQLite datastore 2020-08-26 22:19:19 (sqlite.go:146) -- SQLite event store using file /var/lib/evebox/events.sqlite 2020-08-26 22:19:19 (sqlite.go:60) -- Opening SQLite database /var/lib/evebox/events.sqlite 2020-08-26 22:19:19 (migrator.go:66) -- Current database schema version: 2 2020-08-26 22:19:19 (sqlite.go:94) -- Retention period: 0 days 2020-08-26 22:19:19 (server.go:131) -- Session reaper started 2020-08-26 22:19:19 (purger.go:45) -- Initializing event retention scheduler: period = 0 days 2020-08-26 22:19:19 (server.go:165) -- Authentication disabled. 2020-08-26 22:19:19 (server.go:261) -- Listening on [127.0.0.1]:5636 2020-08-26 22:19:36 (anonymous.go:64) -- Logging in anonymous user {anonymous} from 127.0.0.1:28626 2020-08-26 22:19:37 (alertquery.go:98) -- Alert query time: 437.924µs 2020-08-26 22:19:40 (alertquery.go:98) -- Alert query time: 412.843µs 2020-08-26 22:19:46 (alertquery.go:98) -- Alert query time: 364.432µs 2020-08-26 22:19:52 (alertquery.go:98) -- Alert query time: 354.404µs 2020-08-26 22:19:58 (alertquery.go:98) -- Alert query time: 316.198µs 2020-08-26 22:20:04 (alertquery.go:98) -- Alert query time: 327.127µs 2020-08-26 22:20:10 (alertquery.go:98) -- Alert query time: 387.775µs 2020-08-26 22:20:16 (alertquery.go:98) -- Alert query time: 348.139µs 2020-08-26 22:20:19 (server.go:127) -- Reaping sessions. 2020-08-26 22:20:22 (alertquery.go:98) -- Alert query time: 288.342µs`

evebox.yaml (relevant parts) - `data-directory: /var/lib/evebox http: tls: enabled: false database: type: sqlite input: enabled: true filename: "/var/log/suricata/eve.json" bookmark-directory: /var/lib/evebox rules:

I have added evebox service to firewalld default zone. Port is open - LISTEN 0 65535 127.0.0.1:5636 0.0.0.0:* users:(("evebox",pid=11391,fd=14))

API Access - `# curl -G http://127.0.0.1:5636/api/1/alerts -d time_range=84600s

{"alerts":[],"duration":0}`

Please help.

jasonish commented 4 years ago

Can you verify the formatting of your configuration file? The log file reader is never being started. You should see some log output like:

2020-08-27 10:27:34 (server.go:449) <Info> -- Configuring internal eve log reader
2020-08-27 10:27:34 (bookmarker.go:71) <Info> -- Using bookmark file /var/lib/evebox/b264daf6271f51125d20d5a7715e8947.bookmark
2020-08-27 10:27:34 (bookmarker.go:174) <Info> -- Will start reading at end of file.
2020-08-27 10:27:34 (server.go:261) <Info> -- Listening on [127.0.0.1]:5636
2020-08-27 10:27:37 (evefileprocessor.go:172) <Debug> -- Committed 4 events in 18.884616ms

I'm getting this with your configuration file, which lost its formatting above, but I have it formatted like:

data-directory: /var/lib/evebox

http:
  tls:
    enabled: false

database:
  type: sqlite

input:
  enabled: true
  filename: "/var/log/suricata/eve.json"
  bookmark-directory: /var/lib/evebox
  rules:
    - /var/lib/suricata/rules/*.rules

geoip:
  disabled: false
  database: /var/lib/GeoIP/GeoLite2-City.mmdb

If that still fails, could you try one of the pre-built binaries at https://evebox.org/files/release/0.11.1/?

fluxcap1 commented 4 years ago

Can you verify the formatting of your configuration file? The log file reader is never being started. You should see some log output like:

2020-08-27 10:27:34 (server.go:449) <Info> -- Configuring internal eve log reader
2020-08-27 10:27:34 (bookmarker.go:71) <Info> -- Using bookmark file /var/lib/evebox/b264daf6271f51125d20d5a7715e8947.bookmark
2020-08-27 10:27:34 (bookmarker.go:174) <Info> -- Will start reading at end of file.
2020-08-27 10:27:34 (server.go:261) <Info> -- Listening on [127.0.0.1]:5636
2020-08-27 10:27:37 (evefileprocessor.go:172) <Debug> -- Committed 4 events in 18.884616ms

I'm getting this with your configuration file, which lost its formatting above, but I have it formatted like:

data-directory: /var/lib/evebox

http:
  tls:
    enabled: false

database:
  type: sqlite

input:
  enabled: true
  filename: "/var/log/suricata/eve.json"
  bookmark-directory: /var/lib/evebox
  rules:
    - /var/lib/suricata/rules/*.rules

geoip:
  disabled: false
  database: /var/lib/GeoIP/GeoLite2-City.mmdb

If that still fails, could you try one of the pre-built binaries at https://evebox.org/files/release/0.11.1/?

I apologize for poor formatting of the config file here.

The complete verbose output - evebox server -c /usr/local/evebox/evebox.yaml -v

2020-08-27 20:27:21 (server.go:163) -- This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64 2020-08-27 20:27:21 (server.go:252) -- Self test: found embedded index.html. 2020-08-27 20:27:21 (geoip-service.go:44) -- Failed to initialize geoip database: no database files found 2020-08-27 20:27:21 (configdb.go:59) -- Using configuration database file /var/lib/evebox/config.sqlite 2020-08-27 20:27:21 (migrator.go:66) -- Current database schema version: 1 2020-08-27 20:27:21 (sqlite.go:140) -- Configuring SQLite datastore 2020-08-27 20:27:21 (sqlite.go:146) -- SQLite event store using file /var/lib/evebox/events.sqlite 2020-08-27 20:27:21 (sqlite.go:60) -- Opening SQLite database /var/lib/evebox/events.sqlite 2020-08-27 20:27:21 (migrator.go:66) -- Current database schema version: 2 2020-08-27 20:27:21 (sqlite.go:94) -- Retention period: 0 days 2020-08-27 20:27:21 (server.go:449) -- Configuring internal eve log reader 2020-08-27 20:27:21 (purger.go:45) -- Initializing event retention scheduler: period = 0 days 2020-08-27 20:27:21 (rulemap.go:167) -- Loaded 31633 rules from /var/lib/suricata/rules/suricata.rules 2020-08-27 20:27:21 (rulemap.go:108) -- Loaded 31633 rules 2020-08-27 20:27:21 (server.go:131) -- Session reaper started 2020-08-27 20:27:21 (server.go:165) -- Authentication disabled. 2020-08-27 20:27:21 (bookmarker.go:71) -- Using bookmark file /var/lib/evebox/b264daf6271f51125d20d5a7715e8947.bookmark 2020-08-27 20:27:21 (server.go:261) -- Listening on [127.0.0.1]:5636 2020-08-27 20:27:21 (bookmarker.go:159) -- Found valid bookmark, jumping to offset 8228 2020-08-27 20:27:22 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:22 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:22 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:23 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:23 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:23 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:23 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:24 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:24 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:24 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:24 (evefileprocessor.go:172) -- Committed 1000 events in 105.964µs 2020-08-27 20:27:24 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:24 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:25 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:25 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:25 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:25 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:26 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:26 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:26 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:26 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:26 (evefileprocessor.go:172) -- Committed 1000 events in 109.317µs 2020-08-27 20:27:27 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:27 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:27 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:27 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:27 (indexer.go:107) -- Committing 100 events 2020-08-27 20:27:28 (evefileprocessor.go:172) -- Committed 545 events in 121.415044ms 2020-08-27 20:27:35 (evefileprocessor.go:172) -- Committed 1 events in 89.923024ms 2020-08-27 20:27:43 (evefileprocessor.go:172) -- Committed 1 events in 81.876066ms 2020-08-27 20:27:51 (evefileprocessor.go:172) -- Committed 1 events in 82.215619ms 2020-08-27 20:27:59 (evefileprocessor.go:172) -- Committed 1 events in 90.238949ms 2020-08-27 20:28:07 (evefileprocessor.go:172) -- Committed 1 events in 181.847211ms 2020-08-27 20:28:15 (evefileprocessor.go:172) -- Committed 1 events in 106.727567ms 2020-08-27 20:28:21 (server.go:127) -- Reaping sessions. 2020-08-27 20:28:22 (evefileprocessor.go:182) -- Total: 2551; last minute: 2551; EOFs: 55 2020-08-27 20:28:23 (evefileprocessor.go:172) -- Committed 1 events in 89.693998ms 2020-08-27 20:28:31 (evefileprocessor.go:172) -- Committed 1 events in 90.359562ms 2020-08-27 20:28:40 (evefileprocessor.go:172) -- Committed 1 events in 90.456422ms 2020-08-27 20:28:48 (evefileprocessor.go:172) -- Committed 1 events in 90.16181ms 2020-08-27 20:28:55 (evefileprocessor.go:172) -- Committed 1 events in 99.101715ms 2020-08-27 20:29:03 (evefileprocessor.go:172) -- Committed 1 events in 83.043044ms 2020-08-27 20:29:11 (evefileprocessor.go:172) -- Committed 1 events in 90.554907ms 2020-08-27 20:29:19 (evefileprocessor.go:172) -- Committed 1 events in 98.559342ms evebox_Screenshot_20200827_203027

Still there is no alert on webui.

I tried this binary from your release - https://evebox.org/files/release/0.11.1/evebox-0.11.1-linux-x64.zip but there are no alerts or events in the webui. Details - https://paste.opensuse.org/4f79697e

jasonish commented 4 years ago

Anything when you click on the "Events" view?

fluxcap1 commented 4 years ago

Anything when you click on the "Events" view?

It says no events found.

jasonish commented 4 years ago

Where it says 24 hours in the upper right, drop that down and select "All". Not sure if that will help or not. Usually SQLite doesn't have issues like this.

jasonish commented 4 years ago

Something else to try is one shot mode.

./evebox oneshot /var/log/suricata/eve.json

This uses some hardcoded defaults to should rule out configuration issue, or an issue with the actual program.

My reason for trying my build is I worry if you build locally you might link against an sqlite that is on your system that may not have the correct extensions built in.

fluxcap1 commented 4 years ago

Where it says 24 hours in the upper right, drop that down and select "All". Not sure if that will help or not. Usually SQLite doesn't have issues like this.

Tried it, same thing.

fluxcap1 commented 4 years ago

Something else to try is one shot mode.

./evebox oneshot /var/log/suricata/eve.json

This uses some hardcoded defaults to should rule out configuration issue, or an issue with the actual program.

My reason for trying my build is I worry if you build locally you might link against an sqlite that is on your system that may not have the correct extensions built in.

Did that already but nothing shows in the webui. I tested 4 different browsers but there is no data.

jasonish commented 4 years ago

Browsers won't matter. I'm at a bit of a loss cause this generally just works and is just working for others.

I wonder if its something in your log files? Can you share a portion? You could email them directly to keep them off github.

fluxcap1 commented 4 years ago

Browsers won't matter. I'm at a bit of a loss cause this generally just works and is just working for others.

I wonder if its something in your log files? Can you share a portion? You could email them directly to keep them off github.

Sure, I will provide all the necessary details. Have emailed you eve.json just now.

fluxcap1 commented 4 years ago

Jason,

The issue is resolved after your helpful tips via email. I get alerts and events in webui now, closing this issue. Thank you!