Aiven-Open / http-connector-for-apache-kafka

Apache Kafka Connect sink connector for HTTP
Apache License 2.0
88 stars 44 forks source link

OAuth2 Using Azure Active Directory #32

Open gramanero opened 2 years ago

gramanero commented 2 years ago

I have a proof of concept set up and running on my local machine using the Aiven Http Connector. The idea is to configure the connector to consume data from a kafka topic and then publish the data into an Azure Event Hub. Everything works with one exception, the OAuth request sent to AAD. In order for AAD to respond with a proper access token an additional key/value needs to be added to the request (similar to how we pass along client_id, client_secret, and grant_type). This additional key/value is the resource that the token is being requested for.

I have cloned the Aiven repo, updated the code to support a new property named oauth.client.resource, and have been successful in having AAD return the proper access token such that calls to publish data into Azure Event Hubs works as expected. Without specifying the resource to AAD, an access token is returned however the call to publish the data into Azure Event Hubs is met with a 401 w/ SubCode of 40104: Invalid authorization token audience.

Here is the AAD documentation on the need to supply the resource on the request for the access token: https://docs.microsoft.com/en-us/azure/event-hubs/authorize-access-azure-active-directory

As I understand it, the access token requestor must supply the resource to be accessed and in this case the resource is an Azure Event Hub, but it could be any resource.

My hope is that you consider adding the oauth2.client.resource configuration so that this connector can work securely with Azure resources.

Thank you.

gramanero commented 2 years ago

Wondering if someone can comment on my request. I would like to know if this seems feasible and could be baked into an upcoming release or if I can provide any additional information. Thank you.

willyborankin commented 2 years ago

@gramanero, thank you for the problem you found. I will create an issue for our team.

gramanero commented 2 years ago

@willyborankin Thank you! Happy to assist in any way that I can.

gramanero commented 1 year ago

Wondering if there is any interest in adding support for the feature I requested. I would also be fine to implement it myself on a branch and create a pull request for approval if that would help. Thank you.

julianbergner commented 1 year ago

@gramanero, I am currently facing the same problem. As you mentioned you implemented the new property, would you mind in sharing that code?

gramanero commented 1 year ago

Hi @julianbergner ,

Apologies for the delayed response. I have not looked at the code changes I made since I wrote this post over a year ago. I cracked open the code (yes I kept it locally but we are not using it) and I can try to point you in the right direction. My preference would be to have the the main line of course code updated to support the resource for the access token, but I see no movement on my request.

I am not sure how far the code base I have deviates from the latest release. Meaning, my code changes may not work properly in a newer release. The code changes span two files. search for the word resource in the files and you should be able to see the changes that I made. There "might" be other files that I had changed, but I am not seeing anything offhand so I think it was just these two files.

OAuth2-Resource-Support.zip

Let me know if this works for you.