Dapr OAuth2ClientCredentials Middleware Component is impossible to use

georgestevens99 commented 3 years ago

Expected Behavior

I expect to be able to easily "fill in the blanks" of a component's yml by referring to the Azure documentation and easily getting the information I need. I also expect to have access to examples of filled out yml for all the major providers used by a component, along with references to the vendors document. Further, it would be nice to have code samples, but not necessary.

In regards to the OAuth2 Client Credentials flow in Azure, please note that this is about a service's identity being used to both authenticate access to an Azure Resource, plus to give fine grained RBAC authorization to capabilities of the Azure Resource. Using Identity as a service is currently an Azure Best Practice, including using it with Managed Identities as well. Using a connection string securely stored in a Key Vault is no longer a best practice. Indeed, it is NOT recommended developers do this. However, this is the only option Dapr provides since it is impossible to correctly fill out the yml of the Dapr OAuth Client Credentials Middleware Component.

Here is more useful related information in Dapr Discord posts: (https://discord.com/channels/778680217417809931/778680217417809934/852343001077645312) (https://discord.com/channels/778680217417809931/778680217417809934/852343023073361931)

Furthermore, I really do not understand how one is supposed to set up middle ware components if I am using an Azure Service Bus via Dapr pubsub, and also an Azure Table Storage instance via a Binding/Trigger, etc. Do I put all of these into the same middleware component? Or do I create a separate middleware components for each Azure resource? There is no guidance, documentation nor examples that support cases where a system of collaborating services utilize multiple middleware authN/authZ to access multiple Azure (or other) services. This is required as well to make the current middleware useable. Since this is an Azure best practice Dapr's middleware and supporting docs, yml samples, and code samples must support developers building systems where multiple cloud services are involved, all using OAuth Client Credentials with Azure identity.

Here is something to consider: Perhaps it might be a better developer experience to include OAuth2 Client Credentials flow capabilities in the Dapr component, similar to a secretRef, but rather an identityRef. Thus, each pubsub component could include a reference to the identity based component it wants to use. The same goes for bindings and triggers.

This scopes the identity and its RBAC authorization to a single component, e.g. a pubsub rather than having it in a middle ware component, completely detached from the thing using it for authentication/authz.

Actual Behavior

A few of the "blanks" in the OAuth2 Client Credentials Middleware Component are easy to fill out. Many of them are difficult to figure out what values go in them. There are no examples of filled out yml (that works) for Azure. And, at least for Azure, it seems there is key missing information -- Like the AAD TenantId guid. Thus, for me, this component is unuseable! I had to drop back to implementing the OAuth Client Credentials flow in C# code, and even after that exercise I could not figure out how to fill out the yml.

Please see the attached Microsoft Word document for the specific questions I still have about how to use this component. SvsClientCredMiddlewareComponent2.docx Thanks.

Steps to Reproduce the Problem

Just try filling out the yml. When you have a question, what examples are available to aid you in answering the question? None. What Azure documentation should you view? That is completely unclear.

Release Note

RELEASE NOTE:

AaronCrawfis commented 3 years ago

@yaron2 / @artursouza / @msfussell could you take a look?

artursouza commented 3 years ago

@georgestevens99 Thanks for creating this issue. If I understand the problem correctly, you could not use the OAuth middleware. Is this the documentation you are referring to: https://docs.dapr.io/operations/security/oauth/ ?

I also would like to better understand how the middleware component is related to Dapr pubsub in your application? You can use Dapr pubsub with AzureServiceBus without using the middleware component.

Can you provide a diagram of the scenario you are designing for? By looking at the objective, we can better assist with the problem you are facing.

Thanks, Artur Souza

georgestevens99 commented 3 years ago

Hi Artur - @artursouza

Thanks for your response to my issue. Here are answers to the questions you asked.

You said "you could not use the OAuth middleware." Perhaps a more accurate statement is that I do not understand how to use that middleware. Specifically how to fill in a few of the fields in the yml.

And yes I an referring to the Client Credentials portion of https://docs.dapr.io/operations/security/oauth/

I believe I am close to understanding how to fill in the Client Credentials middleware yml, but there are a few things that I do not understand that are detailed in my MS Word document I attached to component-contrib issue #938, SvsClientCredMiddlewareComponent2.docx .

To summarize, the yml fields that I am having trouble with are: i. tokenURL ii. headerName iii. endpointParamsQuery

You also said "I also would like to better understand how the middleware component is related to Dapr pubsub in your application? You can use Dapr pubsub with AzureServiceBus without using the middleware component."

First, I am an independent contractor and am developing a set of techniques and code that uses Dapr in ways typical to the kinds of client projects I work on, i.e. microservices, mostly. And heavily using messaging as opposed to heavy use of RPC. And also strongly oriented towards Azure. And I always use .NET C#, mostly with ASP.NET Core but not always.

I use both Azure Service Bus queues and topics (mainly topics via Dapr pubsub) and also Azure Storage queues (since they have the capability to have client encryption) for messaging. Also Event Hubs, Blobs, and other Azure services, some of which do not have Dapr components so I write native C# code to utilize them.

I have working samples of Dapr pubsub using Azure Service Bus Topics that uses a connection string. Note that a typical "connection string" in Azure consists of 2 parts, concatenated together: i. The connection string resource identifier. For example: Endpoint=sb://mydaprtestasbns.servicebus.windows.net/;

ii. The connection string Shared Access Signature (SAS) (it can also be other things, but save that for later). For example: SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xFdYZM……………………..abc=

And I have working examples of storing the complete connection string in: i. The pubsub component yml itself (not a good thing to do!), ii. An Environment Variable (less bad, but not so hot!), iii. Also in an Azure Key Vault (the most secure place for connection strings!).

However, in the last couple of years security best practices have undergone a really big change that many people are not aware of. Microsoft now recommends "Do not use connection strings! Even if they are stored in a Key Vault." Why? Because:
i. They are still way to easy for an attacker to get at. ii. When an attacker has a connection string they have a very, very wide attack surface they can exploit. In the case of the Azure Service Bus, the Dapr pubsub component requires a connection string to an entire Service Bus Namespace, that can contain multiple queues and topics. And the connection string gives the holder the right to create and delete queues and topics, as well as read and write from/to them.

The key concept here is the permissions granted by a connection string are NOT fine grained, but coarse grained and thus give an attacker a wide attack surface.

Microsoft's current generation first line of defense is now Identity, rather than connection strings and "the network". This is a major positive change in security best practices that many people are still unaware of, and thus are unknowingly using the outdated previous generation of security measures like connection strings that have a much wider attack surface than exists with the use of Identity.

Using Identity gives each service in your system a unique identity with a provider of Identity-as-a-Service. In our case, that is Azure Active Directory (AAD). Each service is registered with AAD and gets: i. A "clientId", which is a guid, uniquely defining the service (the service is the client of Azure Active Directory). ii. A "clientSecret" or a key to a certificate. (Note that certificates are not currently supported in Dapr Client Credentials, but have been requested in Issue #719).

These are used to authenticate with Azure AD, i.e. a service must present its Id and Secret or CertificateKey.

In addition to registering a service's identity, one must also register the service for specific kinds of things the service is authorized to do with the Azure resource. In other words for each service registered one must also tell AAD the fine grained RBAC permissions granted to that service. P.S. Yesterday it dawned upon me that this area may be what the "endpointParamsQuery" I am having trouble with are concerned with.

For example my sample system has the following fine grained RBAC permissions defined in AAD: i. ServiceA has permission to publish to a single named Service Bus topic. ii. ServiceB has permission to subscribe and receive messages from a single named Service Bus topic.

Please notice how much more constrained the available operations are under Client Credentials versus using a connection string. The attack surface is much, much smaller with Client Credentials.

Also notice that Client Credentials requires an attacker to have at least 2 pieces of information (clientId, clientSecret -- If certificates were supported we'd count the certificate itself as a third piece of info). On the other hand a connection string allows an attacker to have only one piece of info -- the entire connection string -- to gain complete access.

Therefore, the use of Client Credentials middleware is necessary to comply with Microsoft's current recommended practices. Thus, apps with AAD identities are used much like Azure Managed Identities (also an Azure best practice).

The other thing to note is that the clientId and clientSecret or clientCertificateKey are highly confidential secret information, just as is a connection string. Therefore they need to be stored in a Dapr secret, just like a connection string. i. Currently the Dapr middleware clientId and clientSecret are stored in plain text within the middleware component yml. ii. There is no SecretRef capability in the OAuth and Client Credentials middleware that I can see. iii. I will submit an issue for this today (6/14/21) or tomorrow.

You also said "Can you provide a diagram of the scenario you are designing for? By looking at the objective, we can better assist with the problem you are facing." The objective is as stated above -- I need to use Dapr Client Credentials middleware to provide my customers with the security practices currently recommended by Microsoft. So a diagram is not so useful. However, in the above links the Daemon scenario has a nice diagram of how a service uses the Client Credential form of authentication and authorization.

At this point please note that the middleware component yml is completely separate from the Dapr component yml. There are no direct links between them nor does one have any knowledge of the other.

I am not sure of the details of how this middleware works internally, however I suspect that the Azure resource identifier string portion of the connection string in the pubsub component plays a key role. I imagine that the middleware's "scopes" field is involved in recognizing the Azure resource defined in the pubsub component's connection string, but this is just speculation on my part!

In early 2021 I wrote C# code that implements this capability for the Azure Service Bus and that C# code required that I supply the Azure resource identifier portion of the Service Bus Connection string. It could be that the Client Credential component is incomplete, and missing that portion of the connection string. Or perhaps that goes into the "endpointParamsQuery" key/value pairs.

So now you are beginning to see why I am confused -- How does the middleware component know about which resource the Dapr pubsub component is attempting to access? After all the purpose of the client Credential middleware is to allow one service to AuthN/AuthZ with another service, via AAD, and most Azure resources present themselves as services that accept AAD access tokens.

To summarize, the specific problems I am facing are as follows:

How to "fill out" the Dapr Client Credential middleware yml for a single daprized daemon service accessing a single Azure resource/service. Mainly the fields I am having trouble with:
- tokenURL
- headerName
- endpointParamsQuery
How does the middleware component know about the resource the pubsub component is attempting to access. Where does this information go?
How to deal with a single daprized daemon service that accesses multiple Azure services and uses the Dapr Client Credential middleware. It seems to me that a separate Dapr Client Credential middleware component will be needed for each separate Azure service my daemon service needs to access. For example suppose my Daprized ASP.NET gRPC ServiceB needs to: i. Read from an Azure Service Bus Topic subscription. ii. Read some data from a Azure CosmosDB. iii. Then, combine the above data and save it in an Azure Storage Queue. iv. This will require 3 Client Credential middleware components be added to the ServiceB's dapr instance, one for each of the 3 Azure services I am accessing.

Is this correct? Note that all 3 will have the same clientId and clientSecret.

And finally, I do not see a problem with multiple Daprized services (each with their own side cars and their own Client Credential middleware components as required) collaborating together within a single Dapr Mesh. But could there be a problem here that I am not aware of?

I hope the above illustrates the need for improved documentation on how to deal with Client Credential middleware components for: a. How to "fill in" the Client Credentials middleware yaml for a service using a single Azure resource/service, and ensuring the middleware knows which dapr component it is servicing. b. How to deal with single service using multiple Dapr components and hence multiple middleware components. c. Plus, such documentation also needs to include examples that show how the yml of each required middleware component should look like, with all fields filled in.

If there is anything I can do to assist you please let me know.

berndverst commented 3 years ago

I wonder to what extent AAD pod identity addresses this use case. https://github.com/Azure/aad-pod-identity

Here's an example, though it certainly isn't limited to KeyVault: https://docs.dapr.io/reference/components-reference/supported-secret-stores/azure-keyvault-managed-identity/

Independent of that I agree that the middleware needs more documentation and an overhaul.

artursouza commented 3 years ago

@georgestevens99 OAuth middleware does not work with PubSub because the token header is not consumed in the pubsub component. This sample shows how the OAuth middleware can be used in service invocation and might provide better guidance on how to use this component: https://github.com/dapr/samples/tree/master/middleware-clientcredentials

Also, keep in mind that the OAuth middleware is to gain permission to make calls on behalf of the authenticated user, not to the service's own Azure resources.

If you need to authenticate pubsub without using connection strings, we should change the Azure components to support auth via managed identities as well. In this case, you need Azure Service Bus with managed identity. If this sounds what you want to achieve, please, create an issue for that.

georgestevens99 commented 3 years ago

@artursouza Thanks for the quick investigation into this issue. I will definitely check out the link you provided for how to use clientcredentials middleware for Service Invocation. That is a valuable use case for me.

You said "If you need to authenticate pubsub without using connection strings, we should change the Azure components to support auth via managed identities as well. In this case, you need Azure Service Bus with managed identity. If this sounds what you want to achieve, please, create an issue for that." This is definitely what I need to do, but not only with pubsub, also with bindings/triggers and state as well.

I will open an issue for support of client credentials to access services outside a Dapr Mesh using Managed Identities (both system assigned and user assigned) plus unmanaged Azure AD Service Principals, which is what I am talking about in this current issue.

FYI one of the key differences between Managed Identities and unmanaged Azure AD Service Principals is that with Managed Identities (both user and system assigned) Azure takes care of "rolling" the certificates periodically. Plus unmanaged AD Service Principals allow a developer supplied Client Secret rather than Certificate. Otherwise, they are much the same as Managed Identities.

I will write us this issue in the next couple of days, say by Monday 6/21/21 1600 EDT.

Thanks!

dapr-bot commented 3 years ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

dapr-bot commented 3 years ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions.

dapr / components-contrib