Allow an arbitrary OOI declaration to expire by a user specified datetime

originalsouth commented 1 month ago

User Story

As a user, I want allow an arbitrary OOI declaration to expire by a specified datetime, so that OOI declarations can have a lifetime.

As an extension of #3315, allow all type of OOI's to expire (not just end point OOI's like MutedFinding). A lot of the work is actually done in #3315, to make the appropriate changes see:

https://github.com/minvws/nl-kat-coordination/blob/7e1c578ee59b0f0ee72d8c8c319e898d8ba81353/rocky/tools/forms/ooi_form.py#L103-L113

In current main (7b7ae2ef72660ecd4d7348ce4606ffedc16c06d9) with #3315 on top, the garbage collector/event manager (henceforth garbage collector) does not accommodate the expiration of non-endpoint OOI's (as child objects are not removed).

There are various ways one can image to resolve this:

To register the end_valid_time as a property in the OOI's declaration origin, such that when it's OOI is not found the garbage collection logic is activated.
To "test" if the object expired (by looking through the historic information of the object) such that the garbage collection logic is activated.
To assume that a origin with a "dangling" OOI pointer ought to be deleted.

Acceptance Criteria:

[ ] Discuss what the optimal approach is to accommodate object expiration in the garbage collector.
[ ] Activate code in #3315 as provided.
[ ] Introduce said optimal approach in garbage collector.
[ ] The usual testing and finalizing of the patch.

dekkers commented 1 month ago

User Story

As a user, I want allow an arbitrary OOI declaration to expire by a specified datetime, so that OOI declarations can have a lifetime.

Why? This doesn't describe the problem the user wants to solve or the goal the user want to achieve. I think it would be good to start with describing why this feature is needed and how it should work from a user perspective.

originalsouth commented 1 month ago

User Story

As a user, I want allow an arbitrary OOI declaration to expire by a specified datetime, so that OOI declarations can have a lifetime.

Why? This doesn't describe the problem the user wants to solve or the goal the user want to achieve. I think it would be good to start with describing why this feature is needed and how it should work from a user perspective.

An example: suppose I want to scan a website in my existing KAT setup, but I do not want to scan it everyday, just this one time I would like to test it, for the next hour, say, I'll be playing with it.

In comes declare OOI with an expiration date that expires next hour. Run the scans, play with KAT, have fun, and after an hour your playground is cleaned up (for free), without harming your other existing objects.

underdarknl commented 1 month ago

User Story

As a user, I want allow an arbitrary OOI declaration to expire by a specified datetime, so that OOI declarations can have a lifetime.

Why? This doesn't describe the problem the user wants to solve or the goal the user want to achieve. I think it would be good to start with describing why this feature is needed and how it should work from a user perspective.

I feel this is not a very constructive way of expressing your concerns. I believe anyone can come up with a few use-cases for OOI's (especially those manually added) that have an expiration date. Beyond that, being able to is a reasonable feature to want, and might also come in handy with OOI's that have a designated or know EOL date like Certificates, policy documents, or Supported software.

originalsouth commented 1 month ago

There are three proposed in ways we can achieve event management triggering or garbage collection:

Propagation: the end_valid_time gets inherited to all child objects of the parent OOI that has a defined end_valid_time. This way all elements on the branch of the initial OOI expire.
Scheduling: apart from sending the end_valid_time to XTDB a delete-event is recorded, either by the event manager on a queue or, by the scheduler (mula), etc. This way garbage collection is achieved by triggering the delete-event at the appropriate end_valid_time.
Recovering: As the OOI expires within XTDB, octopoes cannot find the object anymore. At the moment it is queried the OOI is "not found" (and we optionally verify it did exist by looking at the objects history) deducing that a delete event ought to be triggered.

minvws / nl-kat-coordination