microsoft / AzureTRE

An accelerator to help organizations build Trusted Research Environments on Azure.
https://microsoft.github.io/AzureTRE
MIT License
182 stars 139 forks source link

Issue with TRE Upgrade to Version 0.17.0 #3911

Closed radms closed 5 months ago

radms commented 5 months ago

When upgrading the TRE to the latest version, the upgrade is failing with the following error:

Error:- Provider produced inconsistent final plan

image
tim-allen-ck commented 5 months ago

Hi @radms have you tried rerunning that job in the pipeline?

radms commented 5 months ago

@tim-allen-ck No, the TRE was deployed and has been manually upgraded to the latest version.

To fix the above issue, the following changes were made to the api-webapp.tf file: https://github.com/microsoft/AzureTRE/blob/main/core/terraform/api-webapp.tf)

image

Once lines 61-63 were commented out, the code ran successfully.

tim-allen-ck commented 5 months ago

Hi @radms that's strange, I've built 0.17 quite a few times and not had to comment those lines out. I have seen that error before though and usually a rerun of terraform fixes it.

radms commented 5 months ago

@tim-allen-ck We tried to rerun the code thrice, but we were still getting the same error on every execution. It only worked for us once we commented out the 3 lines.

tim-allen-ck commented 5 months ago

could you try adding those values to the lifecycle rules, uncomment them and see if it errors?

lifecycle {
    ignore_changes = [
      app_settings["LOGGING_LEVEL", "OTEL_RESOURCE_ATTRIBUTES", "OTEL_EXPERIMENTAL_RESOURCE_DETECTORS"],
    ]
  }
tim-allen-ck commented 5 months ago

Could do with updating the provider versions for core resources. #3919

tim-allen-ck commented 5 months ago

Closing, Please reopen if the problem persists

Danny-Cooke-CK commented 4 months ago

@radms can you confirm the version of TRE you where on before the upgrade ?

radms commented 4 months ago

@Danny-Cooke-CK Before upgrade was using "v0.16.0".

Danny-Cooke-CK commented 4 months ago

For Clarity the underlying issue here was not with the Version 17 release but was actually due to a "make auth" command being run for a second time. This updated the Application secret value's but it doesn't populate them to all necessary resources and hence the environment stopped working.

radms commented 4 months ago

@Danny-Cooke-CK The issue with the 'make auth' command is separate from this one. We are encountering this particular issue during the deployment of the Terraform code.

Danny-Cooke-CK commented 4 months ago

Hi @radms. This ticket was escalated to us and when we spoke with you, you showed us the app secrets had been updated and you didn't understand why but it was causing the deployment to fail ? if thats a different issue then can you raise a different call please