Audit trail

[madelondohmen]

This is an epic ticket

About this epic

Detailed description

Currently within OpenKAT, the "Who did it?" piece is still missing. For example:

• Who added a new user
• Who changed the clearance level

That's why we want a structured log, which people can then filter into. We bring out the JSON, then people can use it for all purposes.

But we also want to process some information in OpenKAT itself, to be able to use this in the future to display in the frontend. Therefore, we store some data from the structured log. (See "Implementation" section for how we can do this.)

Feature benefit/User story

As a user, I want to see who made changes in the organization so that I can detect abuse.

Implementation

Possible solution

There are two paths:

1. Spit out structured JSON when an operation is performed
2. Store in our own data models and possibly make it visible in our frontend

Storing the data

• Where do we store the data? Clearances and OOIs can be given a "user" column in XTDB, in which the "owner" of these objects is put. All objects then belong to a particular user at a particular time. Scheduler jobs and the KATalogus (plugins) must also be linked to a user. So this has to be done in a new database table in the KATalogus. (Actually, you would want the KATalogus to also live in XTDB instead of Postgress. Then you could easily give the owner along with it as well). It is not crazy to write this audit-loggin data to different places. After all, some of the data is functional data (user functionality) and some is not.

  • What data do we store? You may not store much data in the function of logs, but you may store it in the function of functionality.

• How do we store data? UserIDs could be encrypted/hashed.

Excessive logs/alerts.

If you start looking for abuse in data and you link it to names, then you have to deal with the works council, with GDPR, etc.

This is where Kelvin comes in (we're not going to run Elastricsearch, just Kelvin). That looks to see if any outrageous things are happening to a user. In that context, a tool such as Kelvin can automatically detect that something excessive is happening. No person is directly linked to this. But depending on the behavior and risk, you can then pass along a hashed/encrypted userID. Collecting audit logs is a duty, viewing them is a duty, but only if there is a reason. You may also only keep the data for a certain amount of time.

Steps

We need to:

• Invent schema that defines logging (what do you show)
• Write Python library that shows structured logging (JSON)
• Embed and use the library

Scope is now the structured JSON side. But we now need to try to get a picture of what we all want to make insightful.

First logical step:

• Make up schema
• List event codes
• Write Python library that spits out JSON

Alternatives considered

Storing it centrally, but we can't, because the KATalogus is not in XTDB.

Related tickets

• [x] #3043
• [ ] #3044

[underdarknl]

We need to investigate which events we want to add the our Audit Log.

[underdarknl]

We need to create a Concise way of reporting about events based on the use cases of our users.

[underdarknl]

We need to create a Library of method that we can include in the various locations / services that trigger Auditable Events.

[stephanie0x00]

Some inspiration items from the NEN7513 that could also be applicable to our logging are written below:

• CRUDE (create, read, update, delete, execute)

What to log?

• Which event happened?
• Date + time of event
• Which system/process/boefje is affected?
• Which user performed the action?
• Which organisation is affected within OpenKAT? <-- Currently in the audit trail design?
• From where was the action initiated? <-- Currently in the audit trail design?

Logging must be verifiable against the authorisation and access model as applicable when the action was performed.

• Which part of the application did the event occur?
• Which authorisation protocol is applied?
• Which access control model was applicable?

Default actions:

• Creating an organisation/user/ooi/task/other?
• Assignment of IDs
• Data that is entered into the application
• Data additions/updates
• data deletion (users, organisations, oois, tasks, other?)
• Data reading
• data copies and/or printing to paper
• Data that is merged or split
• Data that is transferred to another system or organisation
• Search actions

Special actions:

• Emergency procedures/actions
• Direct access outside of the 'normal' way

Other items to log: Creating/changes/deletions in the authorisation model Enabling/disabling of logging should always be logged. Access to the logging data should always be logged.