Set service endpoint keys passed at runtime

[tdashworth]

Is your feature request related to a problem? Please describe

When deploying a package which includes plugin step registrations, I would like to change the service endpoints (Azure Service Bus Queue) at deployment time for different environments.

Describe the solution you'd like

Provide a dynamic configuration file (defined at deployment time) which can be read and used to modify the service endpoint configuration during the solution import.

[tdashworth]

@ksulikow This was originally raised by you in Azure DevOps and I wanted to check the description above fully encompasses the problem you were having.

[ewingjm]

I think I created this one, @tdashworth. My idea was to pass them as command-line arguments as I think it's the safest option when dealing with credentials.

[tdashworth]

Hey @ewingjm,

Just been looking at this to assign out. Service Endpoints have a number of properties to set although I only think we need to support NamespaceAddress, SolutionNamespace, SASKey, and SASToken. What do you think?

https://docs.microsoft.com/en-us/powerapps/developer/data-platform/reference/entities/serviceendpoint

In terms of the config, I'm thinking:

ServiceEndpointNamespaceAddress:{serviceendpointid} = {new address value} ServiceEndpointSolutionNamespace:{serviceendpointid} = {new namespace value} ServiceEndpointSASKey:{serviceendpointid} = {new key value} ServiceEndpointSASToken:{serviceendpointid} = {new token value}

@mjahlv Have you configured Service Endpoints before? Any thoughts on this approach?

[ewingjm]

At the minute we've got the following parameterised and set per environment -

• namespaceaddress
• solutionnamespace
• saskey

sastoken should also probably be in there as you've pointed out. I can see a case for saskeyname being different between environments although they're the same for us.

[tdashworth]

At the minute we've got the following parameterised and set per environment -

• namespaceaddress
• solutionnamespace
• saskey

sastoken should also probably be in there as you've pointed out. I can see a case for saskeyname being different between environments although they're the same for us.

solutionnamespace added to my previous comment.

[tdashworth]

Just been looking at this page: https://docs.microsoft.com/en-us/powerapps/developer/data-platform/walkthrough-configure-azure-sas-integration#register-a-service-endpoint

Would it be better to replace the different properties with a single connection string that is parsed and sets the respective values?

[tdashworth]

We could also run a verification check calling the bound action TriggerServiceEndpointCheck?

https://docs.microsoft.com/en-us/dynamics365/customer-engagement/web-api/triggerserviceendpointcheck?view=dynamics-ce-odata-9

[ewingjm]

That sounds like it would make a lot of sense. Essentially a mapping between serviceendpointid and a connection string.

[tdashworth]

Okay let's scrap the below:

ServiceEndpointNamespaceAddress:{serviceendpointid} = {new address value} ServiceEndpointSolutionNamespace:{serviceendpointid} = {new namespace value} ServiceEndpointSASKey:{serviceendpointid} = {new key value} ServiceEndpointSASToken:{serviceendpointid} = {new token value}

And go with the this instead:

ServiceEndpoint:{serviceendpointid} = {connection string}

The connection string should be of the format: Endpoint=sb://fourthcoffee.servicebus.windows.net/; SharedAccessKeyName=CrmQueueSharedAccessKey; SharedAccessKey=lxlwDDs5Ct7P/IVjbu2bxROQZgt; EntityPath=myqueue

To parse the connection string we can follow the steps:

1. Split by ;
2. Trim white space from each item
3. Split each item at first = where the first part is the key and the rest is the value

namespaceaddress will equal the endpoint value. e.g sb://fourthcoffee.servicebus.windows.net/ solutionnamspace will equal the first part of the domain from the endpoint e.g. fourthcoffee saskeyname will equal the sharedaccesskeyname value. e.g. CrmQueueSharedAccessKey saskey will equal the sharedaccesskey value. e.g. lxlwDDs5Ct7P/IVjbu2bxROQZgt path will equal the entitypath value. e.g. myqueue

[tdashworth]

Service Endpoints use Solution Layers. This type of modification would be considered an unmanaged change.

We need to verify the managed component is updated accordingly when a new version is deployed.

[satyarkar]

@tdashworth @ewingjm - Anyone working on this issue? With the inclusion of environment variable of type secret is a good option, to store the service endpoint connection string in Azure Key Vault. The runtime setting for this package can be ServiceEndpoint:{serviceendpointid} = {envvariableName}.

[tdashworth]

@satyarkar, no, not currently.

I'm not that clear on your comment above. In the case of ADO, it will already be an environment variable. The docs would look like this:

Set environment variables

You can set service endpoints either through system environment variables (for example, those exposed on Azure Pipelines from your variables or variable groups) or through Package Deployer runtime settings.

Environment variables must be prefixed with PACKAGEDEPLOYER_SETTINGSSERVICEENDPOINT and followed by the service endpoint id. Similarly, runtime settings must be prefixed with EnvVar: and followed by the service endpoint id. For example, if a service endpoint id was 5c9283ba-4197-e911-a990-00224800bb9b, this could be set via either of the following:

Environment variable

$env:PACKAGEDEPLOYER_SETTINGS_SERVICEENDPOINT_5C9283BA-4197-E911-A990-00224800BB9B= "Endpoint=sb://fourthcoffee.servicebus.windows.net/; SharedAccessKeyName=CrmQueueSharedAccessKey; SharedAccessKey=lxlwDDs5Ct7P/IVjbu2bxROQZgt; EntityPath=myqueue"

Import-CrmPackage [...]

Runtime setting

$runtimeSettings = "ServiceEndpoint:5c9283ba-4197-e911-a990-00224800bb9b=Endpoint=sb://fourthcoffee.servicebus.windows.net/; SharedAccessKeyName=CrmQueueSharedAccessKey; SharedAccessKey=lxlwDDs5Ct7P/IVjbu2bxROQZgt; EntityPath=myqueue"

Import-CrmPackage [...] –RuntimePackageSettings $runtimeSettings

The runtime setting takes precedence if both an environment variable and runtime setting are found for the same service endpoint.

---

Now that I write that out, there could be a conflict with the connection string format and runtime settings...

[satyarkar]

@satyarkar, no, not currently.

I'm not that clear on your comment above. In the case of ADO, it will already be an environment variable. The docs would look like this:

Set environment variables

You can set service endpoints either through system environment variables (for example, those exposed on Azure Pipelines from your variables or variable groups) or through Package Deployer runtime settings.

Environment variables must be prefixed with PACKAGEDEPLOYER_SETTINGSSERVICEENDPOINT and followed by the service endpoint id. Similarly, runtime settings must be prefixed with EnvVar: and followed by the service endpoint id. For example, if a service endpoint id was 5c9283ba-4197-e911-a990-00224800bb9b, this could be set via either of the following:

Environment variable

$env:PACKAGEDEPLOYER_SETTINGS_SERVICEENDPOINT_5C9283BA-4197-E911-A990-00224800BB9B= "Endpoint=sb://fourthcoffee.servicebus.windows.net/; SharedAccessKeyName=CrmQueueSharedAccessKey; SharedAccessKey=lxlwDDs5Ct7P/IVjbu2bxROQZgt; EntityPath=myqueue"

Import-CrmPackage [...]

Runtime setting

$runtimeSettings = "ServiceEndpoint:5c9283ba-4197-e911-a990-00224800bb9b=Endpoint=sb://fourthcoffee.servicebus.windows.net/; SharedAccessKeyName=CrmQueueSharedAccessKey; SharedAccessKey=lxlwDDs5Ct7P/IVjbu2bxROQZgt; EntityPath=myqueue"

Import-CrmPackage [...] –RuntimePackageSettings $runtimeSettings

The runtime setting takes precedence if both an environment variable and runtime setting are found for the same service endpoint.

Now that I write that out, there could be a conflict with the connection string format and runtime settings...

@tdashworth - I am saying instead of storing the connection string in the ADO pipeline variable, we will store in the Azure Key Vault and integrate with Power platform environment variable which will be more secure. This is the new feature: (https://docs.microsoft.com/en-us/powerapps/maker/data-platform/environmentvariables#use-azure-key-vault-secrets)

So, the setting will be

$env:PACKAGEDEPLOYER_SETTINGS_SERVICEENDPOINT_5C9283BA-4197-E911-A990-00224800BB9B= "logical name of Power Platform Environment variable".

[tdashworth]

@satyarkar Thanks for the call to discuss this - it now makes sense.

As a recap, you are suggesting an additional option where the Azure Service Bus Connection String is stored in a "Secret" Power Platform Environment Variable (using Azure Service Bus) rather than within Azure DevOps Library Variables. The benefits of this are improved security and segregated storage (in different Azure Key Vaults).

You mentioned that anyone in the system could call the RetrieveEnvironmentVariableSecretValue action to retrieve this value which, upon reflection, I think opens more security concerns than using the ADO secure variables since it could be read by many more people if they knew what they were doing.

Happy to discuss this more 😄

[satyarkar]

@satyarkar Thanks for the call to discuss this - it now makes sense.

As a recap, you are suggesting an additional option where the Azure Service Bus Connection String is stored in a "Secret" Power Platform Environment Variable (using Azure Service Bus) rather than within Azure DevOps Library Variables. The benefits of this are improved security and segregated storage (in different Azure Key Vaults).

You mentioned that anyone in the system could call the RetrieveEnvironmentVariableSecretValue action to retrieve this value which, upon reflection, I think opens more security concerns than using the ADO secure variables since it could be read by many more people if they knew what they were doing.

Happy to discuss this more 😄

@tdashworth - Only the environment variable creator user, access to Azure key vault can be controlled on key vault level, but once the reference of environment variable and azure key vault is established, anyone who have read privileges on environment entity can retrieve the value using action. Agree with you, this will leads to more security concerns (with consideration everyone may have read on environment variable). So for now we will store the connection in the ADO pipeline level.