search

TheHive-Project / TheHive

TheHive: a Scalable, Open Source and Free Security Incident Response Platform
https://thehive-project.org
GNU Affero General Public License v3.0
3.39k stars 618 forks source link

[Enhancement] Integrity check improvement #2334

Closed To-om closed 2 years ago

To-om commented 2 years ago

Request Type

Enhancement

Feature Description

The integrity checks can make TheHive run under heavy load, depending of the amount of data. Currently, each integrity check run randomly every 6 hours. The goal of this issue is to run the integrity checks in a dedicated time slot, configures using cron-like expression. Integrity checks can also be time limited, cancellable and resumable. New APIs are:

POST /api/v1/admin/check/<entityName>/dedup/trigger    Trigger a deduplication
POST /api/v1/admin/check/<entityName>/global/trigger   Trigger a global check
POST /api/v1/admin/check/cancel                        Cancel the current check
GET /admin/index/status                                Returns statistics on checks

New configuration:

integrityCheck.enabled: true  // run integrity check periodically
integrityCheck.schedule: "0 30 2 ? * SUN" // cron expression to trigger an integrity check
integrityCheck.maxDuration: 4 hours // stop integrity check after 4 hours
integrityCheck.integrityCheckConfig.XXX.enabled: true // run integrity check on XXX (Case, Observable, Task, User, ...)
//integrityCheck.integrityCheckConfig.XXX.minTime: 1 minute // minimum time allocated to this integrity check (default: none)
//integrityCheck.integrityCheckConfig.XXX.maxTime: 1 hour // maximum time allocated to this integrity check (default: none)
integrityCheck.integrityCheckConfig.XXX.dedupStrategy: AfterAddition // run dedup check after adding an element (possible values:  AfterAddition, DuringGlobalChecks, AfterAdditionAndDuringGlobalChecks or Disable)
integrityCheck.integrityCheckConfig.XXX.initialDelay: 1 minute // run dedup 1 minute after addition
integrityCheck.integrityCheckConfig.XXX.minInterval: 10 minutes // minimum interval between two dedups