Managed Private Endpoint in Synapse workspace deployment

[stetou]

There is an active issue that was opened on https://github.com/MicrosoftDocs/azure-docs/issues/95661 I think it is relevant to open it here. It is well explained so I just copy/paste here

We have a scenario where we do not want to deploy Development Managed Private Endpoint (MPEs) into different (Test/Prod) as they point to different resources. However, we also do not want to leave orphaned objects in the target that are not in the deployment.

The following settings remove the MPEs and the orphaned objects not in the build.

DeleteArtifactsNotInTemplate: true
DeployManagedPrivateEndpoints: false

The following settings deploy the Development MPEs to Test/Prod which isn't relevant and also removes the existing MPEs and orphaned objects that are not in the build.

DeleteArtifactsNotInTemplate: true
DeployManagedPrivateEndpoints: true

Removing the settings and allowing for the default ignores the MPEs but leaves in place the orphaned objects that are not in the build.

I need this task to Remove the orphaned objects that are not in the build and leave the MPEs alone that already exist in Test/Prod and not deploy the development MPEs.

[lordozb]

Hello @stetou This is a feature request. Kindly raise a support ticket on azure portal.

[stetou]

I did not create a feature request. We now managed the MPEs and here is how we did it for people having the same needs.

We use custom parameters in the workspace template We needed to add some code for the MPEs in template-parameters-definition.json, so we can Override the Parameters during deployment

"Microsoft.Synapse/workspaces/managedVirtualNetworks/managedPrivateEndpoints": {
        "properties": {
            "privateLinkResourceId": "=:-privateLinkResourceId:secureString",
            "groupId": "=",
            "fqdns": "=:-fqdns:secureString"
        }
    }

In the Synapse Deployment option we set these :

DeployManagedPrivateEndpoints: true 
OverrideArmParameters:
mpe-AZDL_privateLinkResourceId : '$(mpe-AZDL_privateLinkResourceId)'
mpe-AZDL_fqdns : '$(mpe-AZDL_fqdns)'
mpe-SQL_privateLinkResourceId : '$(mpe-SQL_privateLinkResourceId)'
mpe-SQL_fqdns : '$(mpe-SQL_fqdns)'
mpe-KeyVault_privateLinkResourceId : '$(mpe-KeyVault_privateLinkResourceId)'
mpe-KeyVault_fqdns : '$(mpe-KeyVault_fqdns)'

That managed 3 MPEs, Datalake, AzureSQLDb and KeyVault

• The MPEs must have the same name in each environments
• You might need a network admin to approve your MPE, that is needed only once

[stetou]

@ericmeans3cloud Thanks for your update. I don't understand how the managed private endpoints get to the template if you don't add it to the Parameter file ...

"you don't even have to add the managed private endpoints to the template parameters file." "the deployment will see that the private endpoint does exist in the template"

Le lun. 8 mai 2023, 15 h 36, ericmeans3cloud @.***> a écrit :

@stetou https://github.com/stetou I ran across this issue and found this thread today, and while working out a solution I found a slightly simpler one - you don't even have to add the managed private endpoints to the template parameters file.

The trick is,

DeleteArtifactsNotInTemplate: true DeployManagedPrivateEndpoints: false

will not delete anything that has the same name as an object that exists in the template, but won't actually update managed private endpoints in the target.

So if you set up "managedprivateendpoint1" in your Dev environment, and "managedprivateendpoint1" in your Prod environment, each pointing to the correct (different) target resource, then if you have DeleteArtifactsNotInTemplate = true, the deployment will see that the private endpoint does exist in the template, so it won't delete it. And since DeployManagedPrivateEndpoints is false, it won't update it in the target environment.

This way you don't have to manage the resource IDs in your overrides; they just get left alone by the deployment.

— Reply to this email directly, view it on GitHub https://github.com/Azure/Synapse-workspace-deployment/issues/73#issuecomment-1538935611, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANSLYGCLBN3UWMOWYAJJ6DXFFDMBANCNFSM6AAAAAASO3V2RE . You are receiving this because you were mentioned.Message ID: @.***>

[ericmeans3cloud]

@stetou - oddly it was working fine up until about May 5th. The managed private endpoints were published in the ARM template, but if you had DeployManagedPrivateEndpoints: false, it wouldn't actually update the target Synapse, so you could manually set up the MPEs in the target and it would neither delete nor modify them. We had this working for a couple of months.

Something changed around May 5th though where the MPEs were no longer being added to the ARM template by default. I don't know what changed or why (we did not make any changes to our pipeline code or how the Synapse is configured).

However, I did discover after some trial and error that telling Synapse to parameterize the MPEs resulted in them being published in the ARM template, but that again as long as you set DeployManagedPrivateEndpoints: false, you don't actually need to override the parameters (it will just leave the target ones alone). Parameterizing them is probably a good idea as it will ensure they get created correctly if you have to redeploy from scratch or they get removed accidentally, but if all you want is to set them up manually and have the deploy not delete them, here's what I did:

1. Add the following to the template-parameters-definition.json file:

   "Microsoft.Synapse/workspaces/managedVirtualNetworks/managedPrivateEndpoints": {
   "properties": {
       "privateLinkResourceId": "-:-privateLinkResourceId",
       "groupId": "-:-groupId",
       "fqdns": "-:-fqdns:array"
   }
   }

2. In Synapse Studio, publish the synapse instance to git.
3. Set up the MPEs you want in the target Synapse. Make sure you use the exact same names for the MPEs in both instances of Synapse (any names in the target that do not exist in the source will be deleted!) However, you can have "extra" MPEs in the source - they will not be created in the target if they do not already exist.
4. Deploy the ARM template with: DeleteArtifactsNotInTemplate: true DeployManagedPrivateEndpoints: false

The MPEs will not be deleted or updated.