VPNGateway with VPNConnection and VPNSiteLink unable to retrieve secret securely (VWAN)

[cedricbraekevelt]

Bicep version Bicep CLI version 0.25.53 (c0ad57dff6)

Describe the bug

• If in VWAN i create my vpnGateway, vpnConnections and vpnSiteLinkConnection in the same vpnGateway object. I'm not able to provide a list of PSK secrets as there is no secure array. I tried to solve this by retrieving the secret in a hashtable via toObject(), but that is not allowed. I'm just not able to retrieve a list of secrets from keyVault which is required for each vpnConnection. I think it's strange that all vpnConnections from this vpnGateway should be using the same PSK.
• If I however declare the vpnConnections as a seperate child resource, in a seperate module, It works, but recreates the vpn's on each deployment (downtime).

If we would be able to use keyvault.getsecret() in the params of a resource this would solve the problem. But at this moment it's only possible as the value of a secure parameter. (or use getsecret everywhere)

resource resourceItem 'Microsoft.Network/vpnGateways@2023-05-01' = {
  name: resourceName
  location: sharedGeneral.location
  properties: {
    bgpSettings: resourceObject.?bgpSettings!
    enableBgpRouteTranslationForNat: resourceObject.?enableBgpRouteTranslationForNat!
    isRoutingPreferenceInternet: resourceObject.?isRoutingPreferenceInternet!
    virtualHub: {
      id: virtualWANHubId
    }
    vpnGatewayScaleUnit: resourceObject.?vpnGatewayScaleUnit! ?? 1
    connections: [for (vpnConnection, i) in resourceObject.?vpnconnections! ?? []:{
        name: resourceNames[i]
        properties: {
          connectionBandwidth: vpnConnection.?connectionBandwidth!
          enableBgp: vpnConnection.?enableBgp!
          enableInternetSecurity: vpnConnection.?enableInternetSecurity!
          enableRateLimiting: vpnConnection.?enableRateLimiting!
          ipsecPolicies: vpnConnection.?ipsecPolicies!
          routingConfiguration: empty(vpnConnection.?routingConfiguration!) ? null : {
            associatedRouteTable: empty(vpnConnection.?routingConfiguration.?associatedRouteTableId!) ? null :{
              id: vpnConnection.?routingConfiguration.?associatedRouteTableId!
            }
            inboundRouteMap: empty(vpnConnection.?routingConfiguration.?inboundRouteMapId!) ? null :{
              id: vpnConnection.?routingConfiguration.?inboundRouteMapId!
            }
            outboundRouteMap: empty(vpnConnection.?routingConfiguration.?outboundRouteMapId!) ? null :{
              id: vpnConnection.?routingConfiguration.?outboundRouteMapId!
            }
            propagatedRouteTables: {
              ids: empty(vpnConnection.?routingConfiguration.?propagatedRouteTables.routeTableIds!) ? null : map(vpnConnection.?routingConfiguration.?propagatedRouteTables.routeTableIds!, routeTableId => {
                  id: routeTableId
                })
              labels: vpnConnection.?routingConfiguration.?propagatedRouteTables.labels!
            }
            vnetRoutes: vpnConnection.?routingConfiguration.?vnetRoutes!
          }
          routingWeight: vpnConnection.?routingWeight!
          trafficSelectorPolicies: vpnConnection.?trafficSelectorPolicies!
          useLocalAzureIpAddress: vpnConnection.?useLocalAzureIpAddress!
          usePolicyBasedTrafficSelectors: vpnConnection.?usePolicyBasedTrafficSelectors!
          vpnConnectionProtocolType: vpnConnection.?vpnConnectionProtocolType!
          remoteVpnSite: {
            #disable-next-line BCP318
            id: vpnSiteList[vpnConnection.vpnSiteName].id
          }
          vpnLinkConnections:  map(vpnConnection.vpnSiteLinkConnections, vpnLinkConnection => {
            name: vpnLinkConnection.vpnSiteLinkName
            properties: {
              connectionBandwidth: vpnLinkConnection.?connectionBandwidth!
              egressNatRules: (vpnLinkConnection.?egressNatRulesIds! == null) ? null : map(vpnLinkConnection.egressNatRulesIds!, natRule => {id: natRule})
              enableBgp: vpnLinkConnection.?enableBgp!
              enableRateLimiting: vpnLinkConnection.?enableRateLimiting!
              ingressNatRules: (vpnLinkConnection.?ingressNatRulesIds! == null) ? null : map(vpnLinkConnection.ingressNatRulesIds!, natRule => {id: natRule})
              ipsecPolicies: vpnLinkConnection.?ipsecPolicies!
              routingWeight: vpnLinkConnection.?routingWeight!
              // sharedKey: keyVault.getSecret('')
              useLocalAzureIpAddress: vpnLinkConnection.?useLocalAzureIpAddress!
              usePolicyBasedTrafficSelectors: vpnLinkConnection.?usePolicyBasedTrafficSelectors!
              vpnConnectionProtocolType: vpnLinkConnection.?vpnConnectionProtocolType!
              vpnGatewayCustomBgpAddresses: vpnLinkConnection.?vpnGatewayCustomBgpAddresses!
              vpnLinkConnectionMode: vpnLinkConnection.?vpnLinkConnectionMode!
              vpnSiteLink: {
                #disable-next-line BCP318
                id: vpnSiteList[vpnConnection.vpnSiteName].siteLinkIds[vpnLinkConnection.vpnSiteLinkName]
              }
            }
          })
        }
      }]
  }
  tags: sharedGeneral.?tags!
}

Am i forgetting something?

As a workaround I'm using a module which outputs the secret in clear text and returns it as an array towards the vpnGateway. The choices were:

• Have PSK's in clear text in deployment pane
• Have PSK's in clear text in Github I chose the former.

[cedricbraekevelt]

Any updates/tips @alex-frankel

[cedricbraekevelt]

@stephaniezyen @alex-frankel am I missing something in my logic or is this just not possible at this moment?

[alex-frankel]

Sorry about the delay here @cedricbraekevelt. This one just kept falling off my radar.

If I however declare the vpnConnections as a seperate child resource, in a seperate module, It works, but recreates the vpn's on each deployment (downtime).

I think this part is the most interesting. connections is another one of those "dual-modelled" properties/child resources and the right long-term fix is to allow redeployment of vpnGateways without affecting the state of connections if connections is NOT declared in the parent resource. I'm going to circulate this with the Networking RP team to see if they are up for this change. Adding @ramandhillon84 as FYI.

In the meantime, I think you have already implemented the "least-bad" solution.

As an aside, can you help us understand the use of both the safe-dereference operator (?) and null-forgiving (!)?

[cedricbraekevelt]

Hi @alex-frankel ,

Thanks for the reply!

The reason I'm using both the ? and ! is because in some edge cases I've had to use them where the output of an expression is used in another expression which resulted in an error. Since them I've began using them both everywhere, maybe not a best practise.

[alex-frankel]

Started a discussion with the VPN Gateway team. Will update you on any responses I get.

[Smoetzak]

Any update on this @alex-frankel?

We are currently doing it like this:

@secure()
param vpnLinkConnection1Secret string
@secure()
param vpnLinkConnection2Secret string

var secrets = [
  vpnLinkConnection1Secret 
  vpnLinkConnection2Secret 
]

....
vpnLinkConnections: [for link, i) in vpnLinkConnections: {
   name: link.name
  properties: {
      ...
      sharedkey: secrets[i]
      ...
   }
}

which is in no way dynamic.