search

microsoft / metered-billing-accelerator

MIT License
40 stars 18 forks source link

Aggregator container app problems #209

Closed mwone-hw closed 7 months ago

mwone-hw commented 8 months ago

Hi, I am experiencing similar issue to #162. I have commented but got no response so I'm opening this new issues. Please see the details in my comment below:

Hi @chgeuer @riccardopinosio, I am encountering similar issues with the same error to the original post has stated. I have read the above thread and realized the solution has to do with setting up the two credentials (which I assume meant app registrations), but would like to ask for further instruction on how to configure them correctly. I am also open to Teams call if it makes life any easier.

Similar to Riccardo, I have also made customization to the deployment scripts similar to what was mentioned in the original post. Here are the modifications I've made to the deployment scripts:

After making the above modification, I attempted the deployment in the following steps:

  1. Set up partner centre offer
  2. Created one app registration and passed ID and secret to the partner centre offer (following the managedAppIntegration guide)
  3. Created new rg hwmeteredbillingsandbox4-rg, location UK South
  4. Assign Blob contributor role to myself (deployer)
  5. Assign contributor role to the app created in step 2
  6. In Powershell console: Az login, select correct subscription, cd to deployment folder, run following deployment script:
    • az deployment group create --template-file main-existing-rg.bicep --parameters appNamePrefix=hwmeteredbillingsandbox4 --parameters ADApplicationID=<Step2AppID> --parameters ADApplicationSecret=<Step2AppSecret> --resource-group hwmeteredbillingsandbox4-rg
    • Provided object ID of the application created in step 2 when prompted for ADObjectID
  7. Encountered error stating "container app environment is not ready for container app creation as it is in state 'Waiting'", resolve by repeating step 6
  8. All deployments are successful, but container app crashes image image

Here is a detailed container app crash log exported from Azure

Many thanks.

Originally posted by @mwone-hw in https://github.com/microsoft/metered-billing-accelerator/issues/162#issuecomment-1941778065

chgeuer commented 8 months ago

Hi @mwone-hw, looking into your CSV export, I see there's a problem getting a token for Azure.Messaging.EventHubs.Amqp.AmqpClient.xxx, i.e. the accelerator cannot talk to EventHubs.

The accelerator expects 2 credentials:

So to me it looks like the instance doesn't successfully get the AZURE_METERING_INFRA_... bits.

In the beginning of this video (https://www.youtube.com/watch?v=WItFCUGwtcQ&list=PLZv8EMzB61uVdnelyxIrVel3EXcGgyMNR&index=9) I go in depth on how these settings play together.

mwone-hw commented 8 months ago

Hi @chgeuer , thank you for the response.

Since I was aiming to use an UAMI for the credential, AZURE_METERING_INFRA_CLIENT_ID, AZURE_METERING_INFRA_CLIENT_SECRET and AZURE_METERING_INFRA_TENANT_ID was left empty as documentation instructed. I understand that it would be best practice to set up the solution with a service principal, but if I was to go down the route of using UAMI instead, is there any specific configuration I should be doing to my (the deployer's) user account?

Thank you again.

chgeuer commented 8 months ago

Hm. You're right, by not setting these 3 variables, the code should be picking up the managed identity... Looking at the docs, this looks correct to me. Need to repro this on ACA...

chgeuer commented 8 months ago

Accidentally hit the "Close" button

mwone-hw commented 8 months ago

Please let me know if there is anything you might need from me to help with reproducing the issue. Thank you.

chgeuer commented 7 months ago

Hi @mwone-hw . Can you try to configure the submission with a 'traditional' service principal, instead of the managed identity?

mwone-hw commented 7 months ago

Hi @chgeuer, thank you for your response.

I've successfully deployed with service principal and the container app did not crash.

The deployment was carried out following the exact steps outlined in the original post, with the only modification being the uncommenting of lines 88-90 in main-existing-rg.bicep.

I'm curious about the reasoning behind the default commenting of the above lines, and which method, using a service principal or UAMI, would be preferred for deployment?

chgeuer commented 7 months ago

Hi @mwone-hw , if I understand correctly, you had to create a dedicated service principal for the "INFRA" credential? I need to repro why in your case in a container app it didn't pick up the managed identity credential.

Conceptually, my view is this: The marketplace credential (to talk to the Azure metering API) might be a long-lived identity in the Entra tenant that is created by somebody in the company, irrespective of the metered-billing-accelerator deployment. So it's kind-of a long-lived credential which is officially registered with marketplace and partner center. The marketplace cred is talking to an outside system, that's why I preferred an explicit service principal, so I don't accidentally delete it when I delete a resource group with the accelerator.

For the infrastructure credential, this is some accelerator-internal thing which ideally should be just a managed identity (if the accelerator compute is running in Azure).