ci: Move away from client-secret in favor of service connections

[ellismg]

Historically we used a client id and client secret pair to authenticate (for both azd itself and other tools it calls, like terraform) in CI. This worked, but the downside of it is that there was a long-lived secret (the client secret) that we had to protect and update.

To mitigate this, our central engineering team is trying to adopt service connections for all our CI jobs. This means that we need to move away from our client-secret based authentication and towards something that uses service connections. For azd itself, it means teaching it how to use something like AzurePipelinesCredential and then updating our jobs to use it.

I'm not sure what the exact answer is for terraform yet. For places we use az we already have an answer via the AzureCLI@2 task which works with service connections.

Since our builds are presently on the floor due to an expired client secret (which we don't want to renew) we'll do this in two parts:

1. Move away from client-secret by delegating auth where we can to the az CLI via the (auth.useAzCliAuth config flag we have) and disabling whatever tests we can't make work quickly with this. This gets our builds green again.

2. Do the work to teach azd how to natively authenticate using a service connection (and figure out how to configure any other tools like terraform to work in this world) and re-enable any tests disabled as part of (1).

[ellismg]

For terraform, I thought that I could be clever and build a ARM_OIDC_REQUEST_URL as outlined in https://registry.terraform.io/providers/hashicorp/azuread/latest/docs/guides/service_principal_oidc#configuring-terraform-to-use-oidc and have that just work for OIDC auth based on what AzurePipelinesCredential, but it did not seem to. I got this error:

cli.go:235: 14.306s [stderr] │ Error: building account: could not acquire access token to parse claims: githubAssertion: received HTTP status 405 with response: {"count":1,"value":{"Message":"The requested resource does not support http method 'GET'."}}

Could be I screwed something up with the URL building however, so I need to dig in more.