evebox with Elasticsearch database

[rickygm]

Hi, I'm using the latest version of Evebox and Elasticsearch. It started recording Suricata data, but then it gives me this error:

evebox[1125]: 2024-11-23 14:15:36 ERROR evebox::eve::processor: Failed to commit events (will try again): elasticsearch commit error: {"errors":true,"too

uid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logstash-2024.11.23","_id":"01JDDA1Y5SN9S4HT5M4848FDJT","status" :409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y5SN9S4HT5M4848FDJT]: version conflict, document already exists (current version [1]) ","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logstash-2024.11.23","_id":"01JDDA1Y5SDJ6ETKFX4RCV05KB ","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y5SDJ6ETKFX4RCV05KB]: version conflict, document already exists (current ve rsion [1])","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logstash-2024.11.23","_id":"01JDDA1Y5SK4TJR2 96B78W73HV","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y5SK4TJR296B78W73HV]: version conflict, document already exists ( current version [1])","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logstash-2024.11.23","_id":"01JDDA 1Y5SQK0JQ5JQ8JSJJENK","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y5SQK0JQ5JQ8JSJJENK]: version conflict, document alread y exists (current version [1])","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logstash-2024.11.23","_i d":"01JDDA1Y5STVPH1H37H4211C5X","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y5STVPH1H37H4211C5X]: version conflict, docum ent already exists (current version [1])","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logstash-2024. 11.23","_id":"01JDDA1Y5TESJVNAYM20GDJDTM","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y5TESJVNAYM20GDJDTM]: version confl ict, document already exists (current version [1])","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}},{"create":{"_index":"logs tash-2024.11.23","_id":"01JDDA1Y7PQMYTBR7X6XVP4Q1P","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[01JDDA1Y7PQMYTBR7X6XVP4Q1P]: ver sion conflict, document already exists (current version [1])","index_uuid":"GHbgCGeVSRaMkIjdIv6RaQ","shard":"0","index":"logstash-2024.11.23"}}}]}

EveBox 0.18.2 elasticsearch 8.16.1 ubuntu 22.04 suricata 7.0.7

any help is apreciated

[jasonish]

Restarting the EveBox process that is writing to Elasticsearch should fix this, until it happens again. I believe we enter this state when some, but not all of the records in a batch submission fail to add, we then retry the whole batch, but some made it in.

Solutions for code are:

• detect which ones were successful, which failed, only retry failed records
• in crease a version number to force overwrite

This is something I'll have to look into.

[rickygm]

I restarted evebox several times, but without success, it always shows the same error and evebox does not show data in the web interface.

A question, how could I increase the number of versions to force writing?

regards!

[rickygm]

I think this is it : evebox elastic set-field-limit 5000