search

Azure / azure-libraries-for-net

Azure libraries for .Net
MIT License
380 stars 192 forks source link

TaskCanceledException creating a service bus #494

Open JadedATB opened 6 years ago

JadedATB commented 6 years ago

In our nightly build, this started failing:

                    serviceBus = await _azure.ServiceBusNamespaces.Define(sb.Name)
                        .WithRegion(region)
                        .WithExistingResourceGroup(rg)
                        .WithSku(sku)
                        .WithNewListenRule("Listener")
                        .WithNewSendRule("Sender")
                        .WithNewManageRule("Manage")
                        .CreateAsync();

Getting a System.Threading.Tasks.TaskCanceledException: Message = "A task was canceled."

stack trace:
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.Azure.Management.ServiceBus.Fluent.NamespacesOperations.<CreateOrUpdateAuthorizationRuleWithHttpMessagesAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ServiceBus.Fluent.NamespacesOperationsExtensions.<CreateOrUpdateAuthorizationRuleAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ServiceBus.Fluent.NamespaceAuthorizationRuleImpl.<CreateChildResourceAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.IndependentChildImpl`7.<CreateResourceAsync>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.ResourceActions.Creatable`4.<Microsoft-Azure-Management-ResourceManager-Fluent-Core-ResourceActions-IResourceCreator<IResourceT>-CreateResourceAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.CreatorTaskItem`1.<ExecuteAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.TaskGroupBase`1.<ExecuteNodeTaskAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.CreatableResources`3.<CreateAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ServiceBus.Fluent.ServiceBusNamespaceImpl.<SubmitChildrenOperationsAsync>d__52.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ServiceBus.Fluent.ServiceBusNamespaceImpl.<CreateResourceAsync>d__57.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.ResourceActions.Creatable`4.<Microsoft-Azure-Management-ResourceManager-Fluent-Core-ResourceActions-IResourceCreator<IResourceT>-CreateResourceAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.CreatorTaskItem`1.<ExecuteAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.TaskGroupBase`1.<ExecuteNodeTaskAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at ######.Azure.Deployment.Deploy.AzureDeploy.<DeployServiceBus>d__195.MoveNext() in F:\Dev\Segment\Automated Deployment\Mainline\Deployment\Deploy\AzureDeploy.ServiceBus.cs:line 130

========================================================== If I simply change the code to be non-async:

                    serviceBus = _azure.ServiceBusNamespaces.Define(sb.Name)
                        .WithRegion(region)
                        .WithExistingResourceGroup(rg)
                        .WithSku(sku)
                        .WithNewListenRule("Listener")
                        .WithNewSendRule("Sender")
                        .WithNewManageRule("Manage")
                        .Create();

It works fine. Why does this happen? Why did this just start happening after working perfectly fine for months before?

JadedATB commented 6 years ago

Update... Using the non-async version didn't totally solve the problem. We still had an issue in our nightly build last night:

#####################################################################

2018-10-19T04:46:08.1720990Z [2018-10-19 04:37:16] Failed to create service bus 'sb-segment-test-ussc-9038': A task was canceled. 2018-10-19T04:46:08.1721174Z [2018-10-19 04:37:16] System.Threading.Tasks.TaskCanceledException: A task was canceled. 2018-10-19T04:46:08.1721346Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1721495Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1721677Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) 2018-10-19T04:46:08.1721822Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ServiceBus.Fluent.NamespacesOperations.d13.MoveNext() 2018-10-19T04:46:08.1722025Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1722162Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1722357Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1722544Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ServiceBus.Fluent.NamespacesOperationsExtensions.d8.MoveNext() 2018-10-19T04:46:08.1722754Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1722947Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1723862Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1724360Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ServiceBus.Fluent.NamespaceAuthorizationRuleImpl.d35.MoveNext() 2018-10-19T04:46:08.1724531Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1724882Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1725144Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1725321Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.IndependentChildImpl`7.d16.MoveNext() 2018-10-19T04:46:08.1725538Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1725708Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1725916Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1726102Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.ResourceActions.Creatable4.<Microsoft-Azure-Management-ResourceManager-Fluent-Core-ResourceActions-IResourceCreator<IResourceT>-CreateResourceAsync>d__15.MoveNext() 2018-10-19T04:46:08.1726343Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1726555Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1726726Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1726942Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.CreatorTaskItem1.d6.MoveNext() 2018-10-19T04:46:08.1727264Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1727411Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1727605Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1727750Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.TaskGroupBase`1.d14.MoveNext() 2018-10-19T04:46:08.1727935Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1728071Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1728251Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1728446Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.CreatableResources3.<CreateAsync>d__2.MoveNext() 2018-10-19T04:46:08.1728591Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1728771Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1728911Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1729099Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ServiceBus.Fluent.ServiceBusNamespaceImpl.<SubmitChildrenOperationsAsync>d__52.MoveNext() 2018-10-19T04:46:08.1729239Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1729530Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1729757Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1729907Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ServiceBus.Fluent.ServiceBusNamespaceImpl.<CreateResourceAsync>d__57.MoveNext() 2018-10-19T04:46:08.1730193Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1730337Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1732616Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1732836Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.ResourceActions.Creatable4.<Microsoft-Azure-Management-ResourceManager-Fluent-Core-ResourceActions-IResourceCreator-CreateResourceAsync>d15.MoveNext() 2018-10-19T04:46:08.1733087Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1733229Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1733439Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1733728Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.CreatorTaskItem`1.d6.MoveNext() 2018-10-19T04:46:08.1734780Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1735525Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1736201Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1737048Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.DAG.TaskGroupBase1.<ExecuteNodeTaskAsync>d__14.MoveNext() 2018-10-19T04:46:08.1737928Z [2018-10-19 04:37:16] --- End of stack trace from previous location where exception was thrown --- 2018-10-19T04:46:08.1738761Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2018-10-19T04:46:08.1739478Z [2018-10-19 04:37:16] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2018-10-19T04:46:08.1740150Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.Extensions.Synchronize[TResult](Func1 function) 2018-10-19T04:46:08.1740851Z [2018-10-19 04:37:16] at Microsoft.Azure.Management.ResourceManager.Fluent.Core.ResourceActions.Creatable`4.Create() 2018-10-19T04:46:08.1741565Z [2018-10-19 04:37:16] at ########.Azure.Deployment.Deploy.AzureDeploy.d__194.MoveNext() in D:\a\9\s\ADSource\Deployment\Deploy\AzureDeploy.ServiceBus.cs:line 140

yaohaizh commented 5 years ago

@JadedATB does this issue still existing with the latest version(1.27.0)?