search

open-telemetry / oteps

OpenTelemetry Enhancement Proposals
https://opentelemetry.io
Apache License 2.0
326 stars 157 forks source link

Sensitive Data Redaction #255

Open svrnm opened 2 months ago

svrnm commented 2 months ago

This OTEP proposes changes that will enable sensitive data redaction. This is a follow up to https://github.com/open-telemetry/semantic-conventions/pull/971 and https://github.com/open-telemetry/semantic-conventions/pull/961 and the SemConv spec meeting (2024-04-29)

svrnm commented 1 month ago

I reworked the document to address (most) of the feedback, please take another look, thanks!

svrnm commented 1 month ago

Overall want to give a HUGE THANK YOU for making a dent in this problem and throwing a proposal out there.

Happy to get this conversation started, a big thank you from me to everyone who provided their feedback so far.

I have a lot of comments. I wish I had more time to give viable solutions to my comments, but I do not. (I also lack the time to be brief, so this is long winded).

I appreciate your feedback, a lot of things I can take and try to provide a solution for.

This has the bones of what I think we want to build.

* A configurable set of redaction rules

* A mechanism to annotate attributes with meta-infromation about sensitivity/security

* A simple user knob of "ON/OFF/my-custom-thing" for important use cases.

💯

The details need a bit more fleshing out in particular:

* The YAML model (and OTEL API) for annotating attributes with baseline sensitivity/redaction controls.  You may need MULTIPLE redaction methods for each level of redaction you support, e.g.

* Stricter lines / definitions (less possible bikeshed) on Sensitivity levels.

* More details on the interaction with the SDK, in particular is it the same interface for all three signals or different?

* More granular model (Span name redaction, e.g.)

I'll take another look into all feedback and try to evolve the document with it.

Overall a great start. Let me know if there's anything I can do to help push areas of this forward or flesh out any of your ideas.

Thanks!

svrnm commented 1 month ago

Thank you all for the great feedback. I was not sure what a good practice is, but I reset the PR to draft because I am in the process of incorporating all the feedback and coming back with a significant change.