[ALM Accelerator - BUG]: export-solution-to-git pipeline fails at step Update Deployment Settings

[fredrikhr]

Describe the issue

I have set up our ALM Accelerator with validation, test and production environments. All three environments specified by their Environment URL (i.e. https://<urlname>.crm<region>.dynamics.com/).

I am exporting a solution from a developer environment. I have set up ALM accelerator to be able to talk to this dev-environment as well (running coe alm connection add ...).

I have looked for the latest release on the repository https://github.com/microsoft/coe-alm-accelerator-templates and found https://github.com/microsoft/coe-alm-accelerator-templates/releases/tag/CenterofExcellenceALMAccelerator-August2022-HF1 (at the time of writing 12 days old). I pushed tags/CenterofExcellenceALMAccelerator-August2022-HF1 into main in our Azure DevOps azure-pipelines repository, so that ALM Accelerator uses the templates from that release.

When pressing the Commit Solution button in the ALM Accelerator for PowerPlatform app the pipeline export-solution-to-git is launched and completes all steps successfully until reaching the step Update Deployment Settings where it fails. From what I can see in the output it fails immediately after tsc has built coe@0.0.1 (presumably when executing the newly built coe tool).

Output (actual environment urls redacted):

> coe@0.0.1 build
> tsc

added 1 package, and audited 3 packages in 520ms

found 0 vulnerabilities
Environments:  validation=https://<...>-validation.crm<...>.dynamics.com/,test=https://<...>-test.crm<...>.dynamics.com/,prod=https://<...>.crm<...>.dynamics.com/
info: Branch start
info: Setup branch
info: Using supplied access token
info: Pipeline Project: undefined
info: An error occurred while creating the branch: Error: {"$id":"1","innerException":null,"message":"A potentially dangerous Request.Path value was detected from the client (:).","typeName":"System.Web.HttpException, System.Web","typeKey":"HttpException","errorCode":0,"eventId":0}
error: {"$id":"1","innerException":null,"message":"A potentially dangerous Request.Path value was detected from the client (:).","typeName":"System.Web.HttpException, System.Web","typeKey":"HttpException","errorCode":0,"eventId":0} {"statusCode":400}
Checking for secrets removed
Secrets removed creating deploymentConfiguration for Validation

    Directory: D:\a\1\s\alm-accelerator-test\TestSolution\config

Mode                 LastWriteTime         Length Name                                                                 
----                 -------------         ------ ----                                                                 
d-----         8/29/2022   6:58 AM                Validation                                                           
Checking for secrets removed
Secrets removed creating deploymentConfiguration for Test
d-----         8/29/2022   6:58 AM                Test                                                                 
Checking for secrets removed
Secrets removed creating deploymentConfiguration for Prod
d-----         8/29/2022   6:58 AM                Prod                                                                 

##[error]PowerShell exited with code '1'.

Expected Behavior

The pipeline export-solution-to-git should complete successfully to enable the subsequent steps in the ALM workflow.

What component are you experiencing the issue with?

ALM Accelerator Pipelines

What solution version are you using?

1.0.20220726.1

Steps To Reproduce

1. Set up CoE environment, install CoE starter kit and ALM Accelerator
2. Set up validation, test and prod environments
3. run coe alm generate install -o install.json, supply values for the validation, test, and prod environments, supply CoE for the managed solution environment. Install all components.
4. run coe alm install -f install.json
5. Set up development environment. Add the ALM accelerator service principal as an S2S App user, create a solution with e.g. one cloud flow in it.
6. run coe alm connection add to create a connection to the dev environment
7. play the ALM Accelerator App, select the dev environment, set up a deployment profile for your solution.
8. Click on the Commit solution button next to your solution.

Anything else?

No response

[mikefactorial]

@fredrikhr this may be related to backward compatibility in Azure DevOps for legacy domains can you validate?

1. Go to your Organization Settings in DevOps
2. Look for the toggle that reads Use the new URL:

[fredrikhr]

@mikefactorial I cannot do that since this would change the URL of an in-production system where my entire organization deploys all of their source code.

[fredrikhr]

I have researched a little bit more and discovered the following:

While running the Update Deployment Settings runs the PowerShell script https://github.com/microsoft/coe-alm-accelerator-templates/blob/CenterofExcellenceALMAccelerator-August2022-HF1/PowerShell/update-deployment-settings.ps1

Ultimately, it calls the New-DeploymentPipelines function defined at https://github.com/microsoft/coe-alm-accelerator-templates/blob/32ebecfccae4cd34e5a699c875427c8786ebe9a2/PowerShell/update-deployment-settings.ps1#L268-L278

In here, the https://github.com/microsoft/coe-starter-kit repository is cloned and checkout at branch main (ref.: https://github.com/microsoft/coe-alm-accelerator-templates/blob/32ebecfccae4cd34e5a699c875427c8786ebe9a2/PowerShell/update-deployment-settings.ps1#L298)

Finally coe alm branch is invoked at https://github.com/microsoft/coe-alm-accelerator-templates/blob/32ebecfccae4cd34e5a699c875427c8786ebe9a2/PowerShell/update-deployment-settings.ps1#L356. But the command only specifies the following parameters:

• -o $orgUrl
• -p "$projectName"
• -r "$repo"
• -d "$solutionName"
• -a $env:SYSTEM_ACCESSTOKEN
• -s $settings (as printed in the log seen in the initial issue description)

The command will ultimately invoke the branch function defined at https://github.com/microsoft/coe-starter-kit/blob/d7a6df107e46a2ca1ec8999a02101282b51eb8b4/coe-cli/src/commands/devops.ts#L836-L873.
Here we can see that the branch function asks for a pipelineProject argument. And sure enough the outut log states:

info: Pipeline Project: undefined

I suspect this issue is related to the pipelineProject argument being unspecified. Note that there has been a change since in the code here since the last release (CoEStarterKit-August2022). The original code for the branch function does not ask for a pipelineProject argument:

https://github.com/microsoft/coe-starter-kit/blob/99f6677c66a2d9a6fd1a91bddfd2e3bd7f56b22e/coe-cli/src/commands/devops.ts#L827-L855

I would like to point out that it might not be safe to simply check-out the main branch in the azure-pipelines template. Wouldn't it be better to checkout a commit or tag that is known to work?

[mikefactorial]

@fredrikhr to your last point, yes we are moving away from pulling down the cli from the main branch and shipping the CLI with the templates (https://github.com/microsoft/coe-alm-accelerator-templates/pull/179) going forward. However, this doesn't seem to be the root cause of the issue you are experiencing. There is a known issue with the backward compatibility of the azure devops url. Can you verify that your organization is using the url backward compatibility flag?

[fredrikhr]

@mikefactorial The toggle is set to NOT use the new URL scheme. I do realize that we have to update to this at some point, however our organization has source code and Azure Package feeds that still rely on the old URL scheme. Therefore, I cannot toggle this setting now immediately since that would affect the entire organization. Are you telling me that there is absolutely no way to deal with this? When I set up Dependabot in organization I ran into a similar issue which I could resolve by setting an environment variable that overrode the DevOps URL that was used. Is there a similar approach I could temporarily use here? I would very much like our organization to start using the ALM workflow in order to be able to promote PowerPlatform and its benefits in good conscience to the organization.

[fredrikhr]

@mikefactorial Alternatively, as I understand it the old URL <orgname>.visualstudio.com will still continue to work, even if I toggle this setting. However, I have not found out for how long this might be the case. Do you have any insights to this (this would possibly help me justify switching to the new URL scheme earlier)?

[mikefactorial]

@fredrikhr Unfortunately, I don't have insight into how or when Azure DevOps will stop allowing the backward compatibility if you toggle it on. Ideally, we would like to add support for this scenario, but it hasn't been raised often so it hasn't been prioritized. If you are able to nail down the specific issue in your investigation and can share that with us via a PR or on this thread, would be happy to review.

[mikefactorial]

Reopening based on regression mentioned in the PR

[fredrikhr]

@mikefactorial I found a new regression today with the newly released CenterofExcellenceALMAccelerator-October2022 tag:

https://github.com/microsoft/coe-alm-accelerator-templates/blob/CenterofExcellenceALMAccelerator-October2022/Pipelines/Templates/update-deployment-settings.yml#L19-L53

Code above re-introduces the problem using hard-coded uri instead of correctly relying on VSTS predefined variables for the team project URL. The statements in the comment is incorrect. The pipeline defines both the pipeline-project and the target repository as pipeline-resource, therefore both can be accessed using pre-defined variables.

[fredrikhr]

Hmm, I actually need this fixed immediately, so I'll get right on it. Expect a PR from me in next hours or so 😁

[mikefactorial]

This has been resolved closing this issue.