Event Hub Namespace Disaster Recovery deployment is not idempotent

[davidbending-rvu]

Bicep version 0.4.613

Describe the bug Deploying event hub namespace disaster recovery config isn't idempotent. The first time the script below runs, it succeeds. The second time it errors instead of doing nothing.

To Reproduce Steps to reproduce the behavior:

var basename = 'eh-zbxbgsg'
var locations = [
  'northeurope'
  'westeurope'
]

resource namespaces 'Microsoft.EventHub/namespaces@2021-01-01-preview' = [for location in locations : {
  name: '${basename}-${location}'
  location: location
  sku: {
    name: 'Standard'
  }
  properties: {
    isAutoInflateEnabled: true
    maximumThroughputUnits: 20
    kafkaEnabled: true
    zoneRedundant: false
  }
}]

resource create_geofailover 'Microsoft.EventHub/namespaces/disasterRecoveryConfigs@2021-01-01-preview' =  {
  name: '${basename}-northeurope/${basename}'
  dependsOn: [
    namespaces
  ]
  properties: {
    partnerNamespace: resourceId('Microsoft.EventHub/namespaces', '${basename}-westeurope')
  }
}

az group create --name db-failoverdemo --location northeurope
az deployment group create --resource-group db-failoverdemo --template-file main.bicep

Additional context I presume this an issue with the underlying API, but I'm not sure where to report that.

[brwilkinson]

Hi @davidbending-rvu

I am able to repro your issue with the following error:

Status Message: Cannot update a namespace which is secondary. For more information visit https://aka.ms/eventhubsarmexceptions.

Following the link I see this is a documented issue.

https://docs.microsoft.com/en-us/azure/event-hubs/resource-manager-exceptions#error-code-badrequest

This is a problem for Declarative style deployments, which I believe you will be forced to implement a workaround within Bicep (based on being a known issue), via a feature flag to note that the secondary has become a geo partner, code sample as below.

Simply switch the value from isSecondary: 0 to isSecondary: 1 after the initial deployment, that will only make changes on the Primary node i.e. the secondary deployment will be skipped (as per the documented recommendation).

var basename = 'eh-zbxbgsg'
var locations = [
  {
    region: 'northeurope'
  }
  {
    region: 'westeurope'
    isSecondary: 0  // once geoconfig has executed once, switch to 1 for redeploys
  }
]

resource namespaces 'Microsoft.EventHub/namespaces@2021-01-01-preview' = [for location in locations: if( ! (contains(location,'isSecondary') && location.isSecondary == 1)) {
  name: '${basename}-${location.region}'
  location: location.region
  sku: {
    name: 'Standard'
  }
  properties: {
    isAutoInflateEnabled: true
    maximumThroughputUnits: 20
    kafkaEnabled: true
    zoneRedundant: false
  }
}]

resource create_geofailover 'Microsoft.EventHub/namespaces/disasterRecoveryConfigs@2021-01-01-preview' = {
  name: 'geoconfig-${namespaces[1].name}'
  parent: namespaces[0]
  dependsOn: [
    namespaces
  ]
  properties: {
    partnerNamespace: namespaces[1].id
  }
}

[davidbending-rvu]

Sorry I didn't explain my problem too well. I know I don't need to create hubs or disaster config on the secondary, because it gets replicated from the primary.

What I'd like to do is have a Bicep file that deploys a pair of replicated event hub namespaces, deploys hubs to them, and configures SAS tokens on the hubs. I'd also like it to be idempotent, so I can have continuous deployment.

My problem is with deploying the disaster config on the primary. It's an error to deploy it if it already exists, and there's no way in Bicep (at least that I know of) to check if it already exists or not so the only workaround I have is to have a deploy_geoconfig flag that I set on a fresh build and clear on subsequent builds. It works, but I was hoping for a cleaner solution.

[brwilkinson]

Yes that is correct.

This is how the provider has been implemented, it's not ideal, hence the workaround.

Did you have a chance to review the sample that I provided?

[davidbending-rvu]

I don't think it's quite right, because it prevents the creation of the namespace in the secondary region but you still need to create that to create the pairing. However, your pattern for using an array of objects containing a secondary flag is much cleaner that what I was doing so I've switched to that pattern for creating the hubs only on the primary namespace.

What I'm now doing is passing a deploy_geofailover flag in from the pipeline and I'll have 2 pipelines - one for a brand new environment and one for CD builds in an existing environment.

[brwilkinson]

The first time you deploy set

isSecondary: 0

Since the 2nd location is not a secondary yet.

Then after the geo has deployed the first time, update the flag to.

isSecondary: 1

Then you can do redeploys.

[brwilkinson]

closing since you have a working scenario. I recommend you revisit the 'feature flag' pattern that I demoed for you, since it's important for many scenarios.

[xavierjohn]

To deploy eventhub to east and west resource group, I use a module. How to access the eventhub namespace in the module from the parent bicep file to pair it?

resource symbolicname 'Microsoft.EventHub/namespaces/disasterRecoveryConfigs@2021-11-01' = {
  name: 'string'
  parent: resourceSymbolicName (How to get the EventHubNamepace that is in a module?)

https://stackoverflow.com/questions/71417723/how-to-setup-eventhub-disasterrecoveryconfigs-using-azure-bicep

[brwilkinson]

Hi @xavierjohn in order to access the parent you can use an existing resource reference.

I don't have a module for eventhub to share however the concept is the same as you see here for CosmosDB

https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/Cosmos-Account-DB.bicep#L5

All you need to make this work is the name of the parent.

resource EventHub 'Microsoft.EventHub/namespaces@2021-11-01' existing = {
  name: '${Deployment}-eh${item.name}'

...

resource symbolicname 'Microsoft.EventHub/namespaces/disasterRecoveryConfigs@2021-11-01' = {
  name: 'string'
  parent: EventHub 

[xavierjohn]

Someone mentioned that solution on stackoverflow but I don’t have an existing resource so it did not work. How to setup EventHub disasterRecoveryConfigs using Azure bicep? - Stack Overflowhttps://stackoverflow.com/questions/71417723/how-to-setup-eventhub-disasterrecoveryconfigs-using-azure-bicep/71417758#71417758

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows

From: Ben @.> Sent: Wednesday, March 9, 2022 9:41 PM To: @.> Cc: @.>; @.> Subject: Re: [Azure/bicep] Event Hub Namespace Disaster Recovery deployment is not idempotent (#4442)

Hi @xavierjohnhttps://github.com/xavierjohn in order to access the parent you can use an existing resource reference.

I don't have a module for eventhub to share however the concept is the same as you see here for CosmosDB

https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/Cosmos-Account-DB.bicep#L5

All you need to make this work is the name of the parent.

resource EventHub @.***' existing = {

name: '${Deployment}-eh${item.name}'

...

resource symbolicname @.***' = {

name: 'string'

parent: EventHub

— Reply to this email directly, view it on GitHubhttps://github.com/Azure/bicep/issues/4442#issuecomment-1063685316, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAOGA7QIKNG3C7Z6H7VN6FDU7GDRRANCNFSM5EB4Y6WA. You are receiving this because you were mentioned.Message ID: @.***>

[brwilkinson]

@xavierjohn I am not sure I follow... ?

If you don't have an existing resource then you will need to define all of the properties to create the resources.