enonic / xp

Enonic XP

https://enonic.com

GNU General Public License v3.0

202 stars 34 forks source link

Add job to clean up audit log #8439

Closed gbbirkisson closed 3 years ago

gbbirkisson commented 4 years ago

Add job that cleans audit logs that are older than X days. Retention set in configuration files.

rymsha commented 4 years ago

We never run periodical tasks (cron like) in XP core before. Even vacuum doesn't run automatically.

It is a good app candidate that would use lib-cron. There are some challenges:

flexible configuration when to run (potentially heavy) cleanup job.
implement correct "delete-by-query" mechanics (Elasticsearch 2.4 doesn't have this feature out of the box). It is OutOfMemory prone.

alansemenov commented 4 years ago

We have added this to our Roadmap. Has to be thoroughly analysed before we begin. Putting this to Backlog and assigning to @rymsha so that it doesn't get lost.

rymsha commented 3 years ago

depend on #8596

rymsha commented 3 years ago

8719 should give s possibility for initial configuration.

We need to come up with decent default config

how often to run cleanup? What time?
how long do we keep data?
enabled or disabled by default?

rymsha commented 3 years ago

Implementation notes:

audit-log-cleanup task descriptor should be located in app-system
it should be able to act on audit-log with millions of records (should use scroll)
it should delete data in chunks, oldest first.