search

opensearch-project / data-prepper

Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.
https://opensearch.org/docs/latest/clients/data-prepper/index/
Apache License 2.0
259 stars 191 forks source link

Using oauth2 or MTLS for authentication #3306

Open mbasha86 opened 1 year ago

mbasha86 commented 1 year ago

Is your feature request related to a problem? Please describe. According to this link https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/otel-trace-source#authentication-configurations , the server supports http basic authentication. We would like to use alternatives i.e oauth2 or MTLS

Describe alternatives you've considered (Optional) Using i.e oauth2 or MTLS

dlvenable commented 1 year ago

We could apply this to other sources as well: otel_logs, otel_metrics, and http.

dlvenable commented 1 year ago

@mbasha86 , This is a great idea. Would either of those options (mTLS or OAuth2) be more valuable to you?

Data Prepper core already uses mTLS for node-to-node communication with the peer-forwarder. So I think we could possibly make use of some of that code and/or configurations.

Would you be interested in working on a PR for this item? We'd be happy to help give some guidance.

mbasha86 commented 1 year ago

Hi @dlvenable

This is a great idea. Would either of those options (mTLS or OAuth2) be more valuable to you?

Yes, security team is looking for alternatives for basic authentication like oauth2 or MTLS.

Would you be interested in working on a PR for this item? We'd be happy to help give some guidance.

No, thanks..I have not used java very long time ago. So, i am afraid that i will not give good results.

KarstenSchnitter commented 1 year ago

We are running DataPrepper behind an NGINX ingress, that provides mTLS. This works nicely. We would prefer mTLS support over OAuth2.

mbasha86 commented 1 year ago

We are running DataPrepper behind an NGINX ingress, that provides mTLS. This works nicely. We would prefer mTLS support over OAuth2.

That's ok for us as well for mTLS per my checking with security team.

KarstenSchnitter commented 1 year ago

If DataPrepper supported OAuth2, would it also forward the tokens to the sinks? That way the user auth could be propagated to the target systems. With end-to-end acknoledgements, this would allow user-specific authorisation over the entire ingestion pipeline.

dlvenable commented 1 year ago

It sounds like the priority would be mTLS and then OAuth2.

If DataPrepper supported OAuth2, would it also forward the tokens to the sinks? That way the user auth could be propagated to the target systems. With end-to-end acknoledgements, this would allow user-specific authorisation over the entire ingestion pipeline.

This is an interesting idea. This could be an optional approach.

mbasha86 commented 1 year ago

If DataPrepper supported OAuth2, would it also forward the tokens to the sinks? That way the user auth could be propagated to the target systems. With end-to-end acknoledgements, this would allow user-specific authorisation over the entire ingestion pipeline.

It will be sufficient for us to have authentication for the entry pipeline (the otel-trace-source input source). However, it's a good idea to have it for the sinks (extra feature) as well if needed.