search

microsoft / coe-starter-kit

Other
751 stars 221 forks source link

[ALM Accelerator - BUG]: [AA4AM] Error "Task was canceled" generated by delete-unmanaged-solution-and-components pipeline #957

Closed Chris-WP closed 3 years ago

Chris-WP commented 3 years ago

Describe the issue

We are experiencing an error when the "delete-unmanaged-solution-and-components" pipeline executes (when triggered from the AA4AM canvas app).

The task "Power Platform Delete Managed Solution" of the "importAndDeleteManaged" job is reporting the following error: ##[error]A task was canceled.

Overall, the pipeline seems to delete the unmanaged solution from the environment as expected, but the pipeline is showing a red failed icon.

Has anyone else experienced the same behaviour?

Expected Behavior

The pipeline should complete without any errors.

What component are you experiencing the issue with?

ALM Accelerator Pipelines

What solution version are you using?

AA4AM - July 2021 Release

Steps To Reproduce

  1. Using the AA4AM canvas app, click to delete the currently installed unmanaged solution (from a Developer Plan Power Platform environment).
  2. The pipeline commences execution
  3. The pipeline displays an error on the 2nd stage of the pipeline
  4. The logs indicate an error within the "Power Platform Delete Managed Solution" task of the "importAndDeleteManaged" job that states "##[error]A task was canceled."
  5. The solution is successfully deleted from the environment (as expected) - even though a pipeline error was reported

Anything else?

Log Output from Power Platform Delete Managed Solution task (with organization and environment identifers removed) is:

2021-08-04T06:29:14.6681603Z ##[section]Starting: Power Platform Delete Managed Solution 2021-08-04T06:29:14.6829259Z ============================================================================== 2021-08-04T06:29:14.6829799Z Task : Power Platform Delete Solution 2021-08-04T06:29:14.6830311Z Description : Power Platform Delete Solution 2021-08-04T06:29:14.6830553Z Version : 0.0.35 2021-08-04T06:29:14.6830818Z Author : Microsoft 2021-08-04T06:29:14.6831334Z Help : More Info 2021-08-04T06:29:14.6831744Z ============================================================================== 2021-08-04T06:29:15.9079566Z VERBOSE: InProcBindingRedirect: 2 - looking to resolve assembly: Microsoft.PowerShell.ConsoleHost.resources, 2021-08-04T06:29:15.9080565Z Version=3.0.0.0, Culture=en-US, PublicKeyToken=31bf3856ad364e35 2021-08-04T06:29:15.9088046Z VERBOSE: InProcBindingRedirect: 1 - looking to resolve assembly: Microsoft.Xrm.WebApi.PowerShell 2021-08-04T06:29:16.0241546Z ##[warning]The names of some imported commands from the module 'Microsoft.Xrm.WebApi.PowerShell' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb. 2021-08-04T06:29:16.4327067Z VERBOSE: InProcBindingRedirect: 3 - looking to resolve assembly: Microsoft.Extensions.DependencyInjection.Abstractions, 2021-08-04T06:29:16.4328627Z Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60 2021-08-04T06:29:16.4391380Z VERBOSE: InProcBindingRedirect: 4 - looking to resolve assembly: System.Threading.Tasks.Extensions, Version=4.2.0.0, 2021-08-04T06:29:16.4392394Z Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 2021-08-04T06:29:16.4450315Z VERBOSE: InProcBindingRedirect: 5 - looking to resolve assembly: Microsoft.Extensions.Logging, Version=3.1.0.0, 2021-08-04T06:29:16.4451125Z Culture=neutral, PublicKeyToken=adb9793829ddae60 2021-08-04T06:29:16.4492470Z VERBOSE: InProcBindingRedirect: 6 - looking to resolve assembly: Microsoft.Extensions.Options, Version=3.1.0.0, 2021-08-04T06:29:16.4493772Z Culture=neutral, PublicKeyToken=adb9793829ddae60 2021-08-04T06:29:16.4559170Z VERBOSE: InProcBindingRedirect: 7 - looking to resolve assembly: Microsoft.Extensions.Logging.Abstractions, 2021-08-04T06:29:16.4560380Z Version=3.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60 2021-08-04T06:29:16.5104930Z VERBOSE: InProcBindingRedirect: 8 - looking to resolve assembly: Microsoft.Extensions.Primitives, Version=3.1.0.0, 2021-08-04T06:29:16.5105766Z Culture=neutral, PublicKeyToken=adb9793829ddae60 2021-08-04T06:29:17.4326383Z **** 2021-08-04T06:29:17.4327646Z Organization details 2021-08-04T06:29:17.4329975Z **** 2021-08-04T06:29:17.4332757Z 8/4/2021 6:29:17 AM | INFO | Connected Org Url: 2021-08-04T06:29:17.4334507Z 8/4/2021 6:29:17 AM | INFO | Connected Org Id: 2021-08-04T06:29:17.4336768Z 8/4/2021 6:29:17 AM | INFO | Connected Org Friendly Name: Chris-WP's Environment 2021-08-04T06:29:17.4338545Z 8/4/2021 6:29:17 AM | INFO | Connected Org Unique Name: 2021-08-04T06:29:17.4342029Z 8/4/2021 6:29:17 AM | INFO | Connected Org Version: 9.2.21072.139 2021-08-04T06:29:17.5882512Z InProcBindingRedirect: resolved requested assembly: Newtonsoft.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed as: Newtonsoft.Json, Version=11.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed 2021-08-04T06:30:58.7621411Z VERBOSE: InProcBindingRedirect: 9 - looking to resolve assembly: System.Management.Automation.resources, 2021-08-04T06:30:58.7622644Z Version=3.0.0.0, Culture=en-US, PublicKeyToken=31bf3856ad364e35 2021-08-04T06:30:58.9328213Z ##[error]A task was canceled. 2021-08-04T06:30:58.9787877Z ##[section]Finishing: Power Platform Delete Managed Solution

devkeydet commented 3 years ago

There's no obvious error message. We typically get more details from the Power Platform Build Tools task than the error you received: 2021-08-04T06:30:58.9328213Z ##[error]A task was canceled.

I've never run into the error above myself. Normally, when this stage fails, we see errors like: _2021-07-23T14:49:14.6202374Z ##[error]Solution dependencies exist, cannot uninstall. DependencyCount : 1 RequiredComponentObject details: Type: Entity, ObjectName: AlmAcceleratorSample, Id: 328bbd4b-88cb-4572-9e8f-0177243f8c0e, ParentObjectName: , ParentId: 00000000-0000-0000-0000-000000000000, DependentComponentObject details: Type: CanvasApp, ObjectName: cat_samplesolutionlistdb0d0, Id: fc805d68-e682-4d0b-823f-8ff55c3d1df4, ParentObjectName: , ParentId: 00000000-0000-0000-0000-000000000000

This is an example of an expected error (yours isn't). There is no actual API to delete an unmanaged solution and all its contents today. So, we do some things in the pipeline to accomplish the scenario that only work under certain conditions. In my example, the Solution System is telling us it's not possible to remove a Solution Component because other Solution Components depend on it. The pipeline has two stages because it is written to be able to rerun in cases like my error example. The expectation, in my error example, is the maker would need to get the Solution in a state where nothing else in the environment depends on the Solution Components inside of it. Once that's the case, under normal circumstances, you should be able to rerun the stage and finish the process.

This is all context to help understand how the pipeline works. None of it solves your specific issue. Back to your issue. I would expect the Power Platform Delete Managed Solution task to provide a more detailed error than what you shared. We don't own the codebase for that task. Another team does. So, you may need to submit a support request for that team. I will also point the team to this GitHub issue for awareness. There are few things you can try before deciding to submit a support request.

  1. Go into maker portal and look at the Solution history. It might give you better error details that are not surfacing through the task.

  2. Since you now have a Managed Solution in your environment, you can try to manually do what the remaining step in the stage does, but in the maker portal. image If it deletes, it was most likely a transient error. If not, then you might have a richer error for us to understand than the task reported in the pipeline. If not, then you could submit a support request to https://admin.powerplatform.microsoft.com/support so the task team can triage the issue. If you want to submit a support request about the Power Platform Build Tools task you should use this option. image

  3. You could also rerun the stage, effectively accomplishing what 2 above does, but you might just get the same unclear error, so I would suggest 2. image If it fails with the same error, then a support request would be advised.

Whatever you do, please do report back. We'll do our best to make sure the right teams have visibility into the fact that you are running into this in the first place.

mikefactorial commented 3 years ago

@Chris-WP Just a hypothesis but it's possible the job timed out waiting for the delete task to complete. Deleting solutions can be a lengthy process even though the default timeout for pipeline executions is pretty high (30 minutes - 60 minutes depending on what license you are using). How long did the job run before it failed?

kkanakas commented 3 years ago

Can you run the pipeline again with Debug mode enabled ?

Debug mode

The output of the pipeline might be a little more verbose

kkanakas commented 3 years ago

Once you have the debug value set to true can see the output in of task looking like this debug task run

Chris-WP commented 3 years ago

Many thanks @devkeydet, @kkanakas, @mikefactorial for the detailed response. It was very helpful.

We are continuing to investigate our end. Some findings so far are:

  1. The error consistently happens each time we attempt to delete the solution (and the pipeline is run)
  2. It doesn't seem to be related to a timeout (as the total pipeline running time is under 10 minutes)
  3. Rerunning the failed job still reports an error
  4. We have now raised a support request about the Power Platform Build Tools (as suggested)

We have rerun the pipeline with system.debug set to true. It provides a bit more information, but not a clear cause of the error:

2021-08-05T09:05:24.6903968Z ##[debug]Delete-Solution:  SolutionName: MyApp_Mobile
2021-08-05T09:05:24.8317165Z InProcBindingRedirect: resolved requested assembly: Newtonsoft.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed as: Newtonsoft.Json, Version=11.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed
2021-08-05T09:07:05.9487728Z VERBOSE: InProcBindingRedirect: 9 - looking to resolve assembly: System.Management.Automation.resources, 
2021-08-05T09:07:05.9488741Z Version=3.0.0.0, Culture=en-US, PublicKeyToken=31bf3856ad364e35
2021-08-05T09:07:06.0077178Z ##[debug]Leaving D:\a\_tasks\PowerPlatformDeleteSolution_a497eae1-e8b2-4ec8-be6a-e865c0b952c1\0.0.35\DeleteSolution.ps1.
2021-08-05T09:07:06.0601653Z ##[debug]Caught exception from task script.
2021-08-05T09:07:06.0631792Z ##[debug]Error record:
2021-08-05T09:07:06.1336792Z ##[debug]Delete-Solution : ************ Exception - solutions(37ca3059-5b1e-4b94-a59c-a66b4fd9e539) : Execute (Delete) request to Dataverse from IOrganizationService |=> A task was canceled.
2021-08-05T09:07:06.1350885Z ##[debug]A task was canceled.
2021-08-05T09:07:06.1367013Z ##[debug]At D:\a\_tasks\PowerPlatformDeleteSolution_a497eae1-e8b2-4ec8-be6a-e865c0b952c1\0.0.35\DeleteSolution.ps1:22 char:19
2021-08-05T09:07:06.1380418Z ##[debug]+ ...     $output = Delete-Solution @Parameters -InformationAction Continue
2021-08-05T09:07:06.1394204Z ##[debug]+                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-08-05T09:07:06.1408416Z ##[debug]    + CategoryInfo          : InvalidOperation: (Microsoft.Power...t.ServiceClient:ServiceClient) [Delete-Solution], T    askCanceledException
2021-08-05T09:07:06.1421285Z ##[debug]    + FullyQualifiedErrorId : ************ Exception - solutions(37ca3059-5b1e-4b94-a59c-a66b4fd9e539) : Execute (Dele    te) request to Dataverse from IOrganizationService |=> A task was canceled.
2021-08-05T09:07:06.1433688Z ##[debug]A task was canceled.,Microsoft.Xrm.WebApi.PowerShell.DeleteSolution
2021-08-05T09:07:06.1454078Z ##[debug]Script stack trace:
2021-08-05T09:07:06.1490990Z ##[debug]at Invoke-DeleteSolution<Process>, D:\a\_tasks\PowerPlatformDeleteSolution_a497eae1-e8b2-4ec8-be6a-e865c0b952c1\0.0.35\DeleteSolution.ps1: line 22
2021-08-05T09:07:06.1504168Z ##[debug]at <ScriptBlock>, D:\a\_tasks\PowerPlatformDeleteSolution_a497eae1-e8b2-4ec8-be6a-e865c0b952c1\0.0.35\DeleteSolution.ps1: line 47
2021-08-05T09:07:06.1518063Z ##[debug]at <ScriptBlock>, <No file>: line 1
2021-08-05T09:07:06.1533473Z ##[debug]at <ScriptBlock>, <No file>: line 22
2021-08-05T09:07:06.1545960Z ##[debug]at <ScriptBlock>, <No file>: line 18
2021-08-05T09:07:06.1560648Z ##[debug]at <ScriptBlock>, <No file>: line 1
2021-08-05T09:07:06.1581446Z ##[debug]Exception:
2021-08-05T09:07:06.1632405Z ##[debug]System.Threading.Tasks.TaskCanceledException: A task was canceled.
2021-08-05T09:07:06.1644876Z ##[debug]   at Microsoft.PowerPlatform.Dataverse.Client.ServiceClient.Execute(OrganizationRequest request)
2021-08-05T09:07:06.1660200Z ##[debug]   at Microsoft.Xrm.WebApi.PowerShell.DeleteSolution.RunOperation(ServiceClient client)
2021-08-05T09:07:06.1673175Z ##[debug]   at Microsoft.Xrm.WebApi.PowerShell.BaseCmdletAuth.ProcessRecord()
2021-08-05T09:07:06.1740977Z ##[error]A task was canceled.
2021-08-05T09:07:06.1743254Z ##[debug]Processed: ##vso[task.logissue type=error]A task was canceled.
2021-08-05T09:07:06.1765422Z ##[debug]Processed: ##vso[task.complete result=Failed]
2021-08-05T09:07:06.2301726Z ##[section]Finishing: Power Platform Delete Managed Solution
Chris-WP commented 3 years ago

We have received the following response from the Microsoft support ticket we have raised:

Based on the error log which you have shared, it is mentioned as below. 

2021-08-06T01:05:54.3851028Z ##[debug]The 'Delete-Solution' command in the Microsoft.Xrm.WebApi.PowerShell' module was imported, but because its name does not include an approved verb, it might be difficult to find. The suggested alternative verbs are "Remove".

Log screenshot

Please refer the doc- https://docs.microsoft.com/en-us/powershell/scripting/developer/cmdlet/approved-verbs-for-windows-powershell-commands?view=powershell-7.1 which explains regarding the verb for the cmdlet that need to be passed.

As per the above screenshot, we could see that Delete or Eliminate verbs should not be used. Instead, you have to use the Remove which is an approved verb.
Chris-WP commented 3 years ago

@devkeydet, @kkanakas, @mikefactorial Is there a way I can adjust the pipelines to use Remove-Solution (rather than Delete-Solution), so I can progress investigation under the Microsoft Support ticket I have raised?

mikefactorial commented 3 years ago

@Chris-WP the response from support is a red herring. The warning in the logs is just a side effect of the naming of the command and has nothing to do with the behavior. There is no Remove-Solution equivalent it's just a semantic warning about the name of the command. Have you tried deleting the solution manually from maker portal as @devkeydet mentioned earlier in the thread? You may get a more helpful error message if you try that (or it may delete without a problem and unblock you). If you have tried from the maker portal and it fails can you download the log file from there and share it?

mikefactorial commented 3 years ago

@Chris-WP closing this issue. Feel free to open another issue if more assistance is required