Not able to provision a service instance within Azure

[hakbl54]

Hi,

Hi created an AKS cluster, When trying to provision a postgreDB I always ends up with a OrphanMitigation

I contacted azure support who was not able to get the issue and redirected me me to your group.

Thanks in advance.

Status for my instance created:

**Name: postgredb6 Namespace: osba Status: OrphanMitigation - Provision call failed: (no description provided) @ 2018-10-03 18:04:05 +0000 UTC Class: azure-postgresql-9-6 Plan: basic

Parameters: location: eastus resourceGroup: hakdeploy**

Bindings: No bindings defined

1. Please find below the steps I followed // Clear the osba and catalogue if previously created helm delete catalog --purge helm delete osba –purge kubectl delete namespaces osba kubectl delete namespace catalog

// install catalog and osba helm repo add svc-cat https://svc-catalog-charts.storage.googleapis.com helm repo add azure https://kubernetescharts.blob.core.windows.net/azure helm install svc-cat/catalog --name catalog --namespace catalog --set rbacEnable=false
helm install azure/open-service-broker-azure --name osba --namespace osba --set azure.subscriptionId=xxx --set azure.tenantId=xxx --set azure.clientId=xxx --set azure.clientSecret=xxx

// provision the postgre db svcat provision postgredb6 --class azure-postgresql-9-6 --plan basic --namespace osba -p location=eastus -p resourceGroup=hakdeploy

kubectl get pods -n catalog Return: NAME READY STATUS RESTARTS AGE catalog-catalog-apiserver-5bd5cbffd7-d8f4c 2/2 Running 0 13m catalog-catalog-controller-manager-77bf5587cb-9vqz6 1/1 Running 6 13m

kubectl get pods -n osba Return: NAME READY STATUS RESTARTS AGE osba-open-service-broker-azure-645b78cb4-cxrns 1/1 Running 2 12m osba-redis-84d6666cf9-fbjdz 1/1 Running 0 12m

[krancour]

We'd need to see Service Catalog controller logs and OSBA logs for a clearer picture, but my gut here says this could be RBAC related. I see you've --set rbacEnable=false when deploying Service Catalog. Recent AKS clusters have RBAC enabled by default, so unless you explicitly disabled that when provisioning the cluster, you don't want to include that option when installing Service Catalog.

If that's not it, post some logs and we'll take another look.

[hakbl54]

Re,

Thanks for your answer, In fact when you create a AKS cluster on azure the Use RBAC is set to No, then if you do not put --set rbacENable=false you got an authorization issue.

logs-from-controller-manager-in-catalog-catalog-controller-manager-77bf5587cb-9vqz6.txt logs-from-open-service-broker-azure-in-osba-open-service-broker-azure-645b78cb4-cxrns.txt

[krancour]

@Harabi I'm certain that new AKS clusters enable RBAC by default, but we can probably set any theory about that being the issue aside...

This bit from the OSBA logs explains what is going on:

time="2018-10-03T18:04:04Z" level=error msg="error executing job; not submitting any follow-up tasks" error="error executing provisioning step \"deployARMTemplate\" for instance \"b1c73f4c-c736-11e8-8be9-6e2eacb3daf0\": error executing provisioning step: error deploying ARM template: error deploying \"6263fea6-250d-4cdc-bd07-abb25448d00f\" in resource group \"hakdeploy\": error getting deployment: resources.DeploymentsClient#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code=\"AuthorizationFailed\" Message=\"The client '820dfc96-b7ec-4165-8822-a5e63f525061' with object id '820dfc96-b7ec-4165-8822-a5e63f525061' does not have authorization to perform action 'Microsoft.Resources/deployments/read' over scope '/subscriptions/b815e902-88c5-4629-92f8-b1affd94c20d/resourcegroups/hakdeploy/providers/Microsoft.Resources/deployments/6263fea6-250d-4cdc-bd07-abb25448d00f'.\"" job=executeProvisioningStep taskID=5ad6e5c1-5d7a-4854-8f42-fcccae2fa56b

The service principal you're using with OSBA doesn't have adequate permissions on the subscription.

[hakbl54]

Do you have an idea about the permissions it needs?

On your doc I can read:

az ad sp create-for-rbac --name osba-quickstart -o table

When creating the cluster from the azure portal, the rbac is set to No by default (this is what I took)

Can I simply create a service principal for rbac and set the rbac usage to false?

[hakbl54]

Hi,

I have more information about it, I set my service principal as contributor to my subscription. My subscription has all the resources required (compute,network and container service). My service principal has the windows active directory permission:

When launching the command I provided you in my first post and looking at the log I do not have the authorization failed. how ever it still failed please find below the logs:

logs-from-open-service-broker-azure-in-osba-open-service-broker-azure-5ff7d9b749-8pj5s.txt logs-from-controller-manager-in-catalog-catalog-controller-manager-77bf5587cb-sw4vq (1).txt

The error now is:

panic: interface conversion: service.InstanceDetails is nil, not *postgresql.allInOneInstanceDetails

The command i type to provision is: svcat provision postgredb6 --class azure-postgresql-9-6 --plan basic --namespace osba -p location=eastus -p resourceGroup=myName

any Idea?

[jeremyrickard]

That appears it might be a bug in the broker. I'll take a look for you @Harabi

[hakbl54]

Hi Jeremy,

I did investigated more on the log side, and on the service catalog controller manager api I can see this:

I1009 04:55:31.927596 1 controller.go:387] Error syncing ServiceInstance osba/postgredb6 (retry: 0/15): The instance references a broker "osba" which has no OSB client created.

In fact after a couple of mins, I can see a postgre db provisioned with an id xxxx which is not the postgredb6 name I defined when I called the command:

svcat provision postgredb6 --class azure-postgresql-9-6 --plan basic --namespace osba -p location=eastus -p resourceGroup=myName

The call to svcat get instances returns nothing. Note that my rbac is disable (default on azure portal website create ressources section) I did not try to activate it after. please find below the log of the svc manager api, the broker log does not contains anything.

logs-from-controller-manager-in-catalog-catalog-controller-manager-f9b8958f8-dsdlp.txt

thanks

[jeremyrickard]

One issue here is that the name you define for kubernetes is not what you will see in the OSBA logs. We are generating a uuid for the name of the resource in azure. The name you gave to svcat is for k8s only.

[jeremyrickard]

when you use “svcat get instances” are you providing a namespace?

[hakbl54]

no

[hakbl54]

C:\Program Files\Microsoft SDKs\Azure.NET SDK\v2.9>svcat get instances -n osba NAME NAMESPACE CLASS PLAN STATUS +------------+-----------+----------------------+-------+--------+ postgredb6 osba azure-postgresql-9-6 basic Ready

[hakbl54]

can see it

[hakbl54]

Seems to have worked. If I summarized the key thing was to add the contributor right of my service principal to my azure subscription. The weird thing is that I do not see the deployment taken into account directly after provisioning. maybe the action is put in a queue.

The other question that I have is regarding the error on the log I sent you:

Error syncing ServiceInstance osba/postgredb6 (retry: 0/15): The instance references a broker "osba" which has no OSB client created.

Is it something normal to have?

Thx