Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.27k stars 3.87k forks source link

A task was canceled #9778

Closed jagdishrajan closed 5 years ago

jagdishrajan commented 5 years ago

I have a PowerShell Azure Function running on a timer trigger hourly. Every so often (perhaps once every 3 days) it throws the following error. Any ideas why? I got this error from Application Insights trace - the Azure Function itself was not executed.

Result: ERROR: A task was canceled.
Exception: A task was canceled.
Stack:    at System.Net.Http.HttpConnection.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)
    at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
    at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
    at Microsoft.Azure.Internal.Subscriptions.SubscriptionsOperations.ListWithHttpMessagesAsync(Dictionary`2 customHeaders, CancellationToken cancellationToken)
    at Microsoft.Azure.Internal.Subscriptions.SubscriptionsOperationsExtensions.ListAsync(ISubscriptionsOperations operations, CancellationToken cancellationToken)
    at Microsoft.Azure.Internal.Subscriptions.SubscriptionsOperationsExtensions.List(ISubscriptionsOperations operations)
    at Microsoft.Azure.Commands.ResourceManager.Common.Paging.GenericPageEnumerator`1..ctor(Func`1 list, Func`2 listNext, UInt64 first, UInt64 skip)
    at Microsoft.Azure.Commands.ResourceManager.Common.Paging.GenericPageEnumerable`1.GetEnumerator()
    at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
    at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
    at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
    at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
    at Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient.ListSubscriptions(String tenantIdOrDomain)
    at Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient.Login(IAzureAccount account, IAzureEnvironment environment, String tenantId, String subscriptionId, String subscriptionName, SecureString password, Boolean skipValidation, Action`1 promptAction, String name, Boolean shouldPopulateContextList)
    at Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.<>c__DisplayClass92_0.<ExecuteCmdlet>b__0(AzureRmProfile localProfile, RMProfileClient profileClient, String name)
    at Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.<>c__DisplayClass94_0.<SetContextWithOverwritePrompt>b__0(AzureRmProfile prof, RMProfileClient client)
    at Microsoft.Azure.Commands.Profile.Common.AzureContextModificationCmdlet.ModifyContext(Action`2 contextAction)
    at Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.SetContextWithOverwritePrompt(Action`3 setContextAction)
    at Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.ExecuteCmdlet()
    at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.<>c__3`1.<ExecuteSynchronouslyOrAsJob>b__3_0(T c)
    at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet, Action`1 executor)
    at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet)
    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Francisco-Gamino commented 5 years ago

@markcowl : What would cause this error Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.ExecuteCmdlet() to happen? This was reported by a customer in Functions.

sphibbs commented 5 years ago

@jagdishrajan can you please provide the following info to help us analyze your issue Description Provide details about the issue you are facing, including when the issue started happening. Has it ever worked as expected? Script/Steps for Reproduction including CLI, PowerShell or SDK where the issue was found Environment PowerShell Get-Module -ListAvailable (Copy past the output) Please run the above script or failing cmdlet with $DebugPreference = ‘Continue’ and paste the resulting debug stream into this ICM issue

jagdishrajan commented 5 years ago

@jagdishrajan can you please provide the following info to help us analyze your issue Description I have a PowerShell Azure Function running on a timer trigger hourly. Every so often (perhaps once every 3 days) it throws the "A task was canceled" error (exception pasted earlier in this thread).

Provide details about the issue you are facing, including when the issue started happening. Has it ever worked as expected? It works most of the time. It's only occasionally, meaning once in 3 days, that this error is thrown and I find it by querying Application Insights.

Script/Steps for Reproduction including CLI, PowerShell or SDK where the issue was found Azure Functions run.ps1 Here's a trimmed down version on the script that I'm now running every 2 minutes to try to reproduce this error. It's not easily reproducible.

param($Timer, $TriggerMetadata)
$DebugPreference = 'Continue'
"FUNCTIONS_EXTENSION_VERSION:{0}  FUNCTIONS_WORKER_RUNTIME:{1}  AZURE_FUNCTIONS_ENVIRONMENT:{2} WEBSITE_TIME_ZONE:{3}" -f $env:FUNCTIONS_EXTENSION_VERSION, $env:FUNCTIONS_WORKER_RUNTIME, $env:AZURE_FUNCTIONS_ENVIRONMENT, $env:WEBSITE_TIME_ZONE
$Host
Get-Module -ListAvailable

Environment Azure Function 2.x FUNCTIONS_EXTENSION_VERSION:~2

PowerShell 6.2.0

Get-Module -ListAvailable (Copy past the output) Please run the above script or failing cmdlet with $DebugPreference = ‘Continue’ and paste the resulting debug stream into this ICM issue


Directory: D:\home\data\ManagedDependencies                                                                                               

ModuleType Version Name PSEdition ExportedCommands


Script 2.5.0 Az Core,Desk
Script 1.6.1 Az.Accounts Core,Desk {Disable-AzDataCollection, Disable-AzContextAutosav…
Script 1.0.0 Az.Advisor Core,Desk {Get-AzAdvisorRecommendation, Enable-AzAdvisorRecom…
Script 1.0.1 Az.Aks Core,Desk {Get-AzAks, New-AzAks, Remove-AzAks, Import-AzAksCr…
Script 1.1.0 Az.AnalysisServices Core,Desk {Resume-AzAnalysisServicesServer, Suspend-AzAnalysi…
Script 1.2.0 Az.ApiManagement Core,Desk {Add-AzApiManagementApiToProduct, Add-AzApiManageme…
Script 1.0.1 Az.ApplicationInsights Core,Desk {Get-AzApplicationInsights, New-AzApplicationInsigh…
Script 1.3.1 Az.Automation Core,Desk {Get-AzAutomationHybridWorkerGroup, Remove-AzAutoma…
Script 1.1.0 Az.Batch Core,Desk {Remove-AzBatchAccount, Get-AzBatchAccount, Get-AzB…
Script 1.0.0 Az.Billing Core,Desk {Get-AzBillingInvoice, Get-AzBillingPeriod, Get-AzE…
Script 1.3.0 Az.Cdn Core,Desk {Get-AzCdnProfile, Get-AzCdnProfileSsoUrl, New-AzCd…
Script 1.2.0 Az.CognitiveServices Core,Desk {Get-AzCognitiveServicesAccount, Get-AzCognitiveSer…
Script 2.4.1 Az.Compute Core,Desk {Remove-AzAvailabilitySet, Get-AzAvailabilitySet, N…
Script 1.0.1 Az.ContainerInstance Core,Desk {New-AzContainerGroup, Get-AzContainerGroup, Remove…
Script 1.1.0 Az.ContainerRegistry Core,Desk {New-AzContainerRegistry, Get-AzContainerRegistry, …
Script 1.1.3 Az.DataFactory Core,Desk {Set-AzDataFactoryV2, Update-AzDataFactoryV2, Get-A…
Script 1.0.0 Az.DataLakeAnalytics Core,Desk {Get-AzDataLakeAnalyticsDataSource, New-AzDataLakeA…
Script 1.2.1 Az.DataLakeStore Core,Desk {Get-AzDataLakeStoreTrustedIdProvider, Remove-AzDat…
Script 1.0.0 Az.DeploymentManager Core,Desk {Get-AzDeploymentManagerArtifactSource, New-AzDeplo…
Script 1.0.0 Az.DevTestLabs Core,Desk {Get-AzDtlAllowedVMSizesPolicy, Get-AzDtlAutoShutdo…
Script 1.1.1 Az.Dns Core,Desk {Get-AzDnsRecordSet, New-AzDnsRecordConfig, Remove-…
Script 1.2.1 Az.EventGrid Core,Desk {New-AzEventGridTopic, Get-AzEventGridTopic, Set-Az…
Script 1.3.0 Az.EventHub Core,Desk {New-AzEventHubNamespace, Get-AzEventHubNamespace, …
Script 1.1.0 Az.FrontDoor Core,Desk {New-AzFrontDoor, Get-AzFrontDoor, Set-AzFrontDoor,…
Script 2.0.0 Az.HDInsight Core,Desk {Get-AzHDInsightJob, New-AzHDInsightSqoopJobDefinit…
Script 1.2.0 Az.IotHub Core,Desk {Add-AzIotHubKey, Get-AzIotHubEventHubConsumerGroup…
Script 1.3.0 Az.KeyVault Core,Desk {Add-AzKeyVaultCertificate, Update-AzKeyVaultCertif…
Script 1.3.0 Az.LogicApp Core,Desk {Get-AzIntegrationAccountAgreement, Get-AzIntegrati…
Script 1.1.0 Az.MachineLearning Core,Desk {Move-AzMlCommitmentAssociation, Get-AzMlCommitment…
Script 1.0.0 Az.ManagedServices Core,Desk {Get-AzManagedServicesAssignment, New-AzManagedServ…
Script 1.0.0 Az.MarketplaceOrdering Core,Desk {Get-AzMarketplaceTerms, Set-AzMarketplaceTerms}
Script 1.1.0 Az.Media Core,Desk {Sync-AzMediaServiceStorageKey, Set-AzMediaServiceK…
Script 1.2.1 Az.Monitor Core,Desk {Get-AzMetricDefinition, Get-AzMetric, Remove-AzLog…
Script 1.12.0 Az.Network Core,Desk {Add-AzApplicationGatewayAuthenticationCertificate,…
Script 1.1.0 Az.NotificationHubs Core,Desk {Get-AzNotificationHub, Get-AzNotificationHubAuthor…
Script 1.3.2 Az.OperationalInsights Core,Desk {New-AzOperationalInsightsAzureActivityLogDataSourc…
Script 1.1.2 Az.PolicyInsights Core,Desk {Get-AzPolicyEvent, Get-AzPolicyState, Get-AzPolicy…
Script 1.1.0 Az.PowerBIEmbedded Core,Desk {Remove-AzPowerBIWorkspaceCollection, Get-AzPowerBI…
Script 1.4.3 Az.RecoveryServices Core,Desk {Get-AzRecoveryServicesBackupProperty, Get-AzRecove…
Script 1.1.0 Az.RedisCache Core,Desk {Remove-AzRedisCachePatchSchedule, New-AzRedisCache…
Script 1.0.1 Az.Relay Core,Desk {New-AzRelayNamespace, Get-AzRelayNamespace, Set-Az…
Script 1.6.1 Az.Resources Core,Desk {Get-AzProviderOperation, Remove-AzRoleAssignment, …
Script 1.3.0 Az.ServiceBus Core,Desk {New-AzServiceBusNamespace, Get-AzServiceBusNamespa…
Script 1.1.1 Az.ServiceFabric Core,Desk {Add-AzServiceFabricApplicationCertificate, Add-AzS…
Script 1.0.2 Az.SignalR Core,Desk {New-AzSignalR, Get-AzSignalR, Get-AzSignalRKey, Ne…
Script 1.13.1 Az.Sql Core,Desk {Get-AzSqlDatabaseTransparentDataEncryption, Get-Az…
Script 1.5.1 Az.Storage Core,Desk {Get-AzStorageAccount, Get-AzStorageAccountKey, New…
Script 1.2.0 Az.StorageSync Core,Desk {Invoke-AzStorageSyncCompatibilityCheck, New-AzStor…
Script 1.0.0 Az.StreamAnalytics Core,Desk {Get-AzStreamAnalyticsFunction, Get-AzStreamAnalyti…
Script 1.0.1 Az.TrafficManager Core,Desk {Add-AzTrafficManagerCustomHeaderToEndpoint, Remove…
Script 1.4.0 Az.Websites Core,Desk {Get-AzAppServicePlan, Set-AzAppServicePlan, New-Az…

Directory: D:\Program Files (x86)\SiteExtensions\Functions\2.0.12590\32bit\workers\powershell\Modules                                     

ModuleType Version Name PSEdition ExportedCommands


Binary 0.1.0 Microsoft.Azure.Functions.PowerShe… Core {Push-OutputBinding, Get-OutputBinding, Trace-Pipel…
Manifest 1.1.0.0 Microsoft.PowerShell.Archive Desk {Compress-Archive, Expand-Archive}
Manifest 6.1.0.0 Microsoft.PowerShell.Management Core {Add-Content, Clear-Content, Clear-ItemProperty, Jo…
Manifest 6.1.0.0 Microsoft.PowerShell.Utility Core {Export-Alias, Get-Alias, Import-Alias, New-Alias…}
Script 1.4.3 PackageManagement Desk {Find-Package, Get-Package, Get-PackageProvider, Ge…
Script 1.1.7.0 PackageManagement Desk {Find-Package, Get-Package, Get-PackageProvider, Ge…
Script 1.6.7 PowerShellGet Desk {Find-Command, Find-DSCResource, Find-Module, Find-…
Binary 1.1.2 ThreadJob Desk Start-ThreadJob

jagdishrajan commented 5 years ago

I haven't seen this issue using the simple script above in 3 days, so I'm closing it.

wallism commented 10 months ago

I was getting this error with az.storage 6.1.0 with this command: $context = New-AzStorageContext -StorageAccountName accountname which works when you include the key! $context = New-AzStorageContext -StorageAccountName accountname -StorageAccountKey accountkey

(would be nice if the error was a little more helpful)