🚀 Feature: Add separate audit log event stream

schultzp2020 commented 3 months ago

🔖 Feature description

This feature aims to add an audit log event stream which involves implementing a dedicated system for tracking and recording critical security-related events and actions within Backstage. This separate event stream ensures that security-sensitive activities such as user authentication, authorization, data access, and configuration changes are logged independently from regular operational logs. By maintaining a distinct audit log event stream, organizations can enhance security, facilitate compliance with regulatory requirements, enable efficient incident response, and uphold the integrity of critical audit data.

🎤 Context

Importance of Audit Logging:

Enhance Security: Tracking critical security-related events such as user authentication, authorization, and data access bolsters the protection of your Backstage instance against unauthorized activities.
Ensure Compliance: Fulfill regulatory requirements by logging configuration changes and other security-sensitive operations, aiding your organization in adhering to industry standards.
Enable Investigation: Detailed event tracking facilitates forensic analysis and investigation of security incidents, empowering administrators to swiftly identify and respond to security breaches.
Maintain Integrity: Implementing restricted access and robust tamper-proof measures upholds the integrity of audit logs, ensuring the reliability and accuracy of recorded events.

Why Separate Audit Logging?

Security and Integrity: Segregating audit logs ensures higher security measures, such as restricted access and encryption, safeguarding sensitive information from unauthorized tampering or access.
Regulatory Compliance: Compliance frameworks necessitate organizations to maintain audit logs for security and accountability purposes. Separating audit logs from normal logs aids in managing and verifying compliance requirements more efficiently.
Event Filtering and Analysis: Independent audit log event streams facilitate filtering and analysis of security-related events, enabling efficient monitoring, detection, and investigation of security incidents without interference from non-security-related events.
Performance and Scalability: Optimizing performance and scalability, separating audit logs allows dedicated resources and configurations tailored specifically for the requirements of audit logging, capturing critical events with higher frequency and granularity.
Clear Differentiation: Separating audit logs from normal logs helps distinguish security-related events from other operational events, simplifying troubleshooting, debugging, and monitoring processes.

In summary, maintaining a separate event stream for audit logging ensures that security-critical events are handled with appropriate security measures, compliance adherence, and efficiency, while also optimizing performance and clarity in event management and analysis.

✌️ Possible Implementation

The only necessary adjustments involve expanding Backstage's Winston configuration to accommodate a distinct audit log event stream: link.

For instance, you can refer to this implementation: link.

By implementing this change, users can maintain a separate audit log event stream while retaining the existing normal log event stream. For example:

// This message is exclusively recorded in the audit log event stream
logger.log({ level: 'info', message: 'This is an audit log message', isAuditLog: true })

// This message remains unchanged and appears in the normal log event stream
logger.log({ level: 'info', message: 'This is a log message' })

👀 Have you spent some time to check if this feature request has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?

Yes I am willing to submit a PR!

schultzp2020 commented 3 months ago

I'm a bit unsure about how the Winston implementation works in Backstage. Which specific file(s) should I edit to make adjustments? I noticed that createRootLogger is referenced in the current backend. However, WinstonLogger is utilized in dynamicPluginsRootLoggerServiceFactory, designed for the dynamic backend. Is there a way to streamline this setup to facilitate easier modifications in the future? I feel like the current implementation is overly complicated and confusing. Ideally, I believe we should remove WinstonLogger, a wrapper class, and stick with createRootLogger.

kunickiaj commented 3 months ago

In an attempt to answer this -- there's the legacy backend (what is linked first) and then there's the new backend system (not linked) and I think the dynamic backend is something under development.

With the "new" backend system (default in current releases) you can replace the default logging service implementation but you'll have to use backend.createSpecializedBackend(services) with your service factory implementations (e.g. for the root logger or other logging). rather than backend.create() which uses all the default service factories.

But also +1 to a dedicated audit log (or even just another logger).

tudi2d commented 3 months ago

Thanks for raising this issue - it's been coming up from time-to-time, but there does not exist something like this yet. Just want to refer to some ongoing/older discussions here to have the context available:

schultzp2020 commented 2 months ago

Comments from @Rugvip on discord.

https://discord.com/channels/687207715902193673/687235481154617364/1227301762634088581

We're moving over to the LoggerService being the way to log: https://backstage.io/docs/backend-system/core-services/logger. The WintsonLogger is the basis for the default implementation of that, but it's ofc possible to switch out

https://discord.com/channels/687207715902193673/687235481154617364/1227303938705981492

I find it to be very similar to structured logging, just even more structured. I don't think it needs a ton of frameworkification, just some good handling of users etc. You might for example let it consume the new credentials objects and derive fields from the principals that executed an action.

Prolly worth having it be a separate interface from the logger service though, but then likely have a default implementation that just forwards it to the logger.

Hard part is what kind of information plugins are supposed to provide at this point imo, and what level of guarantees we want thay particular actions were actually executed, and the granularity.

Bit hard to prioritize against a lot of the other things going on too so I don't think it's realistically something that can be added in the next few months tbh.

Rugvip commented 1 month ago

Quick update on this and how to move it forward. I think it makes sense to add as such, but needs more refinement as discussed above. Next step would be a BEP that starts outlining goals and suggesting a concrete implementation.

schultzp2020 commented 1 month ago

Quick update on this and how to move it forward. I think it makes sense to add as such, but needs more refinement as discussed above. Next step would be a BEP that starts outlining goals and suggesting a concrete implementation.

Great! I will start working on creating a BEP.

backstage / backstage