Azure / logicapps

Azure Logic Apps labs, samples, and tools
MIT License
358 stars 299 forks source link

Logicapps Throws an exception in the portal: System.Linq: Sequence contains no elements. ; Runtime version: error #725

Closed rob-thijssen closed 1 year ago

rob-thijssen commented 1 year ago

Describe the Bug

When deploying OR restarting an existing logicapp in the azure portal, the logicapp won't start and throws an exception "System.Linq: Sequence contains no elements." image

The runtime version information shows "error" image

 

Following log was extracted from the diagnostig dump (\LogFiles\Application\Functions\Host)

2023-02-06T18:34:48.918 [Error] A host error has occurred during startup operation '686eb671-2baf-4bd0-a59a-23393efa53d4'.
System.InvalidOperationException : Sequence contains no elements
   at System.Linq.ThrowHelper.ThrowNoElementsException()
   at System.Linq.Enumerable.Max[TSource,TResult](IEnumerable`1 source,Func`2 selector)
   at Microsoft.Azure.WebJobs.Script.ScriptHost.AddFunctionDescriptors(IEnumerable`1 functionMetadata) at /_/src/WebJobs.Script/Host/ScriptHost.cs : 553
   at async Microsoft.Azure.WebJobs.Script.ScriptHost.InitializeFunctionDescriptorsAsync(IEnumerable`1 functionMetadata,CancellationToken cancellationToken) at /_/src/WebJobs.Script/Host/ScriptHost.cs : 527
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.ScriptHost.InitializeAsync(CancellationToken cancellationToken) at /_/src/WebJobs.Script/Host/ScriptHost.cs : 314
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.ScriptHost.StartAsyncCore(CancellationToken cancellationToken) at /_/src/WebJobs.Script/Host/ScriptHost.cs : 258
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation,Int32 attemptCount,JobHostStartupMode startupMode) at /_/src/WebJobs.Script.WebHost/WebJobsScriptHostService.cs : 311
2023-02-06T18:34:48.944 [Information] Stopping JobHost

I Found the following differences in some logging today..

2023-02-02T09:19:53.820 [Information] Host Status: {
  "id": "<removed>",
  "state": "Running",
  "version": "4.14.0.19631",
  "versionDetails": "4.14.0+fc14864b4fc2a095f8cd92c7f0ca338dbeabd430",
  "platformVersion": "99.0.10.793",
  "instanceId": "a80b1f0da498d6ce334d9c16deb626b8ee48592f64e056dc5eca61cf7318c7cf",
  "computerName": "pd0sdwk000P94",
  "processUptime": 148800861,
  "functionAppContentEditingState": "Unknown",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle.Workflows",
    "version": "1.3.7"
  }
}
2023-02-06T16:06:09.751 [Information] Host Status: {
  "id": "<removed>",
  "state": "Error",
  "version": "4.15.1.19904",
  "versionDetails": "4.15.1+7570b6daba86ae58931f116694e78700f4e86cd4",
  "platformVersion": "99.0.10.793",
  "instanceId": "a80b1f0da498d6ce334d9c16deb626b8ee48592f64e056dc5eca61cf7318c7cf",
  "computerName": "pd0sdwk000P94",
  "errors": [
    "System.Linq: Sequence contains no elements."
  ],
  "processUptime": -3182079,
  "functionAppContentEditingState": "Unknown",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle.Workflows",
    "version": "1.3.7"
  }
}

Could this mean that the version 4.15.1.19904 is causing troubles?

Update: This issue also occurs on all newly deployed logicapps AND for logicapps created on the portal when the Confguration Value FUNCTIONS_WORKER_RUNTIME is changed from node to dotnet

Plan Type

WS1

Steps to Reproduce the Bug or Issue

Goto the portal Select an existing Logicapps Standard resource Restart the Logicapp

Workflow JSON

No response

Screenshots or Videos

No response

Additional context

CoreCLR Version: 6.0.922.41905 .NET Version: 6.0.9 Description: The process was terminated due to an unhandled exception. Exception Info: System.AggregateException: One or more errors occurred. (One or more errors occurred. (Object reference not set to an instance of an object.)) ---> System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.) ---> System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.Azure.WebJobs.Script.Grpc.AspNetCoreGrpcServer.ShutdownAsync() in //src/WebJobs.Script.Grpc/Server/AspNetCoreGrpcServer.cs:line 56 at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcInitializationService.OuterStopAsync(CancellationToken cancellationToken) in //src/WebJobs.Script/Workers/Rpc/RpcInitializationService.cs:line 72 at Microsoft.Azure.WebJobs.Script.HostedServiceManager.StopAsync(CancellationToken cancellationToken) in //src/WebJobs.Script/HostedServiceManager.cs:line 29 at Microsoft.AspNetCore.Hosting.HostedServiceExecutor.ExecuteAsync(Func2 callback, Boolean throwOnFirstFailure) --- End of inner exception stack trace --- at Microsoft.AspNetCore.Hosting.HostedServiceExecutor.ExecuteAsync(Func2 callback, Boolean throwOnFirstFailure) at Microsoft.AspNetCore.Hosting.WebHost.StopAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Hosting.WebHostExtensions.WaitForTokenShutdownAsync(IWebHost host, CancellationToken cancellationToken) at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage) at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage) at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at Microsoft.Azure.WebJobs.Script.WebHost.Program.Main(String[] args) in //src/WebJobs.Script.WebHost/Program.cs:line 31

AB#17030685

g2udevelopment commented 1 year ago

We have the same on all restarted Logic Apps in a standard plan. Hope to bump this one.

goran-nultien commented 1 year ago

++

MaurodeLyon commented 1 year ago

Yeah we've encountered the same issue when utilizing Logic Apps (Standard)

g2udevelopment commented 1 year ago

@rob-thijssen there is a workaround, put this in your configuration FUNCTIONS_EXTENSION_VERSION = 4.14.0.19631 , to hard pin the version, this resolved the issue for us.

rob-thijssen commented 1 year ago

@rob-thijssen there is a workaround, put this in your configuration FUNCTIONS_EXTENSION_VERSION = 4.14.0.19631 , to hard pin the version, this resolved the issue for us.

@g2udevelopment -> You are a genius! I wonder how you came up with this solution as a WORKAROUND! (?) I tested it, and it also resolves the breaking issue for us. For now my team and I at least can continue working on new and existing logicapps in DEVELOPMENT.

But this certainly is NOT the solution for production as far as I'm concerned. I hope the LogicApps team can fix the issue so we can still use the latest version whenever available. That is one of the reasons for using PAAS services from Microsoft, right?

tilicon commented 1 year ago

We've seen this as well, the workaround for us was to remove the FUNCTIONS_WORKER_RUNTIME setting.

rohithah commented 1 year ago

This was caused by a regression, and we are actively working on a hotfix to mitigate the issue, and the workaround suggested by @g2udevelopment should work ( We have notified the subscription owners of the impacted applications with the same workaround.).

rob-thijssen commented 1 year ago

Hello @rohithah, I notice that there is a new version (4.15.2.20177) deployed on our subscription for the runtime version on all LogicApps that I restarted. Does this mean that this issue is now officially fixed, and I can safely revert the workaround for the FUNCTIONS_EXTENSION_VERSION by @g2udevelopment on all production LogicApps?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 45 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 14 days since being marked as stale.