Deploying a new SF App + Service via ARM template fails (Invalid template: Circular dependency)

[johncrim]

I have not been able to successfully deploy an ARM template for a new SF app containing both a Microsoft.ServiceFabric/clusters/applications resource and a Microsoft.ServiceFabric/clusters/applications/services resource, as documented here.

Repro steps:

1. Start with a new Service Fabric cluster on Azure with linux VMs
2. Create an ARM template for deploying an SF app to the cluster (or better: Create an ARM template for an SF app with user-assigned managed identity to the cluster)
   • I used API version 2019-06-01-preview, since I’m using managed identity, eg:

     "type": "Microsoft.ServiceFabric/clusters/applications",
     "apiVersion": "2019-06-01-preview"

   • Ensure that the ARM template contains both a Microsoft.ServiceFabric/clusters/applications resource and a Microsoft.ServiceFabric/clusters/applications/services resource, where the services resource describes the SF service within the application.
3. Deploy the ARM template – you should receive an Invalid template error, similar to:

   Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'Circular dependency detected on resource: '/subscriptions/<subid>/resourceGroups/<resourceGroup>/providers/Microsoft.ServiceFabric/clusters/(clusterName)/applications/(applicationName)'. Please see https://aka.ms/arm-template/#resources for usage details.'.

I have tried all of the following options for the "dependsOn": block:

1. No specified dependency from the service to the application (fails with "ParentResourceNotFound")
2. Specify the dependency from the service to the application by resource name (this typically works in ARM templates, but seems to fail here because the resource name includes a ‘/’ ). The error is:

   Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The template resource 'testme-stage/APPNAME/SERVICENAME' at line '174' and column '9' is not valid: The resource identificator 'testme-stage/APPNAME' is malformed. Please see https://aka.ms/arm-template-expressions/#reference for usage details.. Please see https://aka.ms/arm-template-expressions for usage details.'.

3. Specify the dependency from the service to the application by resource id, as shown in the documentation. The error is:

   Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'Circular dependency detected on resource: '/subscriptions/subscriptionId/resourceGroups/stage/providers/Microsoft.ServiceFabric/clusters/testme-stage/applications/APPNAME'. Please see https://aka.ms/arm-template/#resources for usage details.'.

The below file “ServiceFabricApplication.jsonc” contains comments showing all 3 options, none of which work. I have also tried specifying the service resource as a child resource of the application (which seems like the best model), but it also fails with the circular dependency error.

The only workaround I can find is to omit the Microsoft.ServiceFabric/clusters/applications/services resource; another workaround might be a 2 step deploy (deploy 1 ARM template containing the Microsoft.ServiceFabric/clusters/applications resource; the deploy a 2nd ARM template containing the a Microsoft.ServiceFabric/clusters/applications/services resource) – I have not tried this. But this is clearly a significant bug, because what the documentation directs developers to do does not work.

Here's an ARM template that fails to deploy (though any ARM template containing a services resource should fail):

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "clusterName": {
      "type": "string",
      "defaultValue": "[concat('testme-', resourceGroup().name)]",
      "metadata": {
        "description": "Name of the SF cluster"
      }
    },
    "clusterLocation": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location of the SF cluster"
      }
    },
    "applicationTypeName": {
      "type": "string",
      "metadata": {
        "description": "Name of the SF application type being deployed"
      }
    },
    "applicationTypeVersion": {
      "type": "string",
      "metadata": {
        "description": "Version of the SF application being deployed"
      }
    },
    "applicationName": {
      "type": "string",
      "defaultValue": "[parameters('applicationTypeName')]",
      "metadata": {
        "description": "Name of the SF application being deployed"
      }
    },
    "sasTokenQueryString": {
      "type": "string",
      "metadata": {
        "description": "Azure storage SAS token + query string for temporary access to the .sfpkg file in packages blob storage."
      }
    },
    "statelessServiceName": {
      "type": "string",
      "metadata": {
        "description": "Name of the stateless service"
      }
    },
    "serviceIdentityName": {
      "type": "string",
      "defaultValue": "[parameters('statelessServiceName')]",
      "metadata": {
        "description": "Name of the ManagedIdentity the service uses"
      }
    }
  },
  "variables": {
    "appTypeVersionResourceId": "[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applicationTypes/', parameters('applicationTypeName'), '/versions/', parameters('applicationTypeVersion'))]",
    "appResourceName": "[concat(parameters('clusterName'), '/', parameters('applicationName'))]",
    "appResourceId": "[resourceId('Microsoft.ServiceFabric/clusters/applications', parameters('clusterName'), parameters('applicationName'))]",
    "appPackageUrl": "[concat('https://testmepackages.blob.core.windows.net/', toLower(parameters('applicationTypeName')), '/', parameters('applicationTypeVersion'), '/', parameters('applicationTypeName'), '-', parameters('applicationTypeVersion'), '.sfpkg', parameters('sasTokenQueryString'))]",
    "serviceIdentityResourceId": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', parameters('serviceIdentityName'))]"
  },
  "resources": [
    { // Create the managed service identity
      "name": "[parameters('serviceIdentityName')]",
      "type": "Microsoft.ManagedIdentity/userAssignedIdentities",
      "apiVersion": "2018-11-30",
      "location": "[parameters('clusterLocation')]"
    },
    {
      "name": "[concat(parameters('clusterName'), '/', parameters('applicationTypeName'))]",
      "type": "Microsoft.ServiceFabric/clusters/applicationTypes",
      "apiVersion": "2019-03-01",
      "location": "[parameters('clusterLocation')]",
      "dependsOn": [],
      "properties": {
      }
    },
    {
      "name": "[concat(parameters('clusterName'), '/', parameters('applicationTypeName'), '/', parameters('applicationTypeVersion'))]",
      "type": "Microsoft.ServiceFabric/clusters/applicationTypes/versions",
      "apiVersion": "2019-03-01",
      "location": "[parameters('clusterLocation')]",
      "dependsOn": [
        "[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applicationTypes/', parameters('applicationTypeName'))]"
      ],
      "properties": {
        "appPackageUrl": "[variables('appPackageUrl')]"
      }
    },
    { // Create the application
      "name": "[variables('appResourceName')]",
      "type": "Microsoft.ServiceFabric/clusters/applications",
      "apiVersion": "2019-06-01-preview",
      "location": "[resourceGroup().location]",
      "dependsOn": [
        "[variables('appTypeVersionResourceId')]",
        "[parameters('serviceIdentityName')]"
      ],
      "identity": {
        "type": "userAssigned",
        "userAssignedIdentities": {
          "[variables('serviceIdentityResourceId')]": {}
        }
      },
      "properties": {
        "typeName": "[parameters('applicationTypeName')]",
        "typeVersion": "[parameters('applicationTypeVersion')]",
        "managedIdentities": [
          {
            "name": "[parameters('serviceIdentityName')]",
            "principalId": "[reference(variables('serviceIdentityResourceId'), '2018-11-30').principalId]"
          }
        ],
        "upgradePolicy": {
          "forceRestart": "false",
          "rollingUpgradeMonitoringPolicy": {
            "failureAction": "Rollback",
            "healthCheckWaitDuration": "00:01:00.0",
            "healthCheckStableDuration": "00:01:00.0",
            "healthCheckRetryTimeout": "00:02:00.0",
            "upgradeTimeout": "00:05:00.0",
            "upgradeDomainTimeout": "00:02:00.0"
          },
          "applicationHealthPolicy": {
            "considerWarningAsError": "false",
            "maxPercentUnhealthyDeployedApplications": "50",
            "defaultServiceTypeHealthPolicy": {
              "maxPercentUnhealthyServices": "50"
            }
          }
        }
      }
    }
    ,{
      "name": "[concat(parameters('clusterName'), '/', parameters('applicationName'), '/', parameters('statelessServiceName'))]",
      "type": "Microsoft.ServiceFabric/clusters/applications/services",
      "apiVersion": "2019-03-01",
      "location": "[parameters('clusterLocation')]",
      "properties": {
        "serviceKind": "Stateless",
        "serviceTypeName": "[parameters('statelessServiceName')]",
        "instanceCount": "-1",
        "partitionDescription": {
          "partitionScheme": "Singleton"
        }
      },
      "dependsOn": [
        "[variables('appTypeVersionResourceId')]"
        // Don't include either of the following, for:
        // New-AzResourceGroupDeployment : 12:17:29 PM - Resource Microsoft.ServiceFabric/clusters/applications/services 'testme-stage/APPNAME/SERVICENAME' failed with message '{
        //         "error": {
        //           "code": "ParentResourceNotFound",
        //           "message": "Put operation on resource with no parent is not allowed. Resource: /subscriptions/subscriptionId/resourcegroups/STAGE/providers/Microso
        // ft.ServiceFabric/clusters/testme-STAGE/applications/APPNAME/services/SERVICENAME. Parent: /subscriptions/subscriptionId/resourcegroups/STAGE/provide
        // rs/Microsoft.ServiceFabric/clusters/testme-STAGE/applications/APPNAME.",
        //     "details": []
        //         }
        //       }'
        //
        // Include the following, for:
        //  Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The template resource 'testme-stage/APPNAME/SERVICENAME' at line '174' and column '9' is not valid: The resource identificator 'testme-stage/APPNAME' is malformed. Please see https://aka.ms/arm-template-expressions/#reference for usage details.. Please see https://aka.ms/arm-template-expressions for usage details.'.
        //"[variables('appResourceName')]"
        //
        // Include the following, for:
        //  Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'Circular dependency detected on resource: '/subscriptions/subscriptionId/resourceGroups/stage/providers/Microsoft.ServiceFabric/clusters/testme-stage/applications/APPNAME'. Please see https://aka.ms/arm-template/#resources for usage details.'.
        //"[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applications/', parameters('applicationName'))]"
      ]
    }
  ],
  "outputs": {
    "application": {
      "type": "object",
      "value": "[reference(variables('appResourceId'))]"
    }
  }
}

[amenarde]

The service should depend on the application resource, not on the applicationtypeversion resource. Please see this sample template from the managed identity walk-through, and see if this resolves your issues.

https://github.com/Azure-Samples/service-fabric-managed-identity/blob/master/ResourceManagement/app.template.json

[johncrim]

Thank you for the reply @amenarde . Just to clarify, the error I was getting when using the dependency on the application resource was:

        // Include the following, for:
        //  Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The template resource 'testme-stage/APPNAME/SERVICENAME' at line '174' and column '9' is not valid: The resource identificator 'testme-stage/APPNAME' is malformed. Please see https://aka.ms/arm-template-expressions/#reference for usage details.. Please see https://aka.ms/arm-template-expressions for usage details.'.
        //"[variables('appResourceName')]"

I have not tested this again since posting the issue, but the workaround of omitting the Microsoft.ServiceFabric/clusters/applications/services resource is working acceptably. I wouldn't be surprised if this problem still exists, but it doesn't seem to be bothering many other people.