jasonish / evebox

Web Based Event Viewer (GUI) for Suricata EVE Events in Elastic Search
https://evebox.org/
MIT License
431 stars 67 forks source link

sqlite/fts: disk space failed to be returned #279

Closed jasonish closed 1 year ago

jasonish commented 1 year ago

With the new SQLite FTS (full-text search) introduced in 0.17.0, database retention has an issue where the indexed FTS data is not freed as it should be. EveBox will still delete events by date, or disk usage, however, less and less of that space becomes available for events.

0.18.0 has fixed this issue, but it does require that FTS be re-indexed which can be done by disabling FTS and re-enabling with the following commands:

evebox sqlite fts disable ~/path/to/events.sql
evebox sqlite fts enable ~/path/to/events.sql

Note: You should make sure EveBox is not running, and add the --force flag.

jasonish commented 1 year ago

Turns out after a few inserts, etc, the FTS tables sort themselves out. Or can be forced with the following SQLite command:

INSERT INTO ft(ft) VALUES('optimize');
jasonish commented 1 year ago

Resolved by adding a the sqlite fts optimize command. But normally this won't be needed unless you are aggressively trying to shrink the size of your database file.