Open kelps opened 2 years ago
I experienced a similar issue but mine is related to using a Linux App Service Plan for the Azure Function with .NET 6 isolated
Mine is running on Windows, both local and on Azure. The previous version os this function was .net5. The only difference between the 2 are the migration to .net6 and the use of some C# new features.
Did you check the Log stream (verbose) on your Function App? because at least for me it showed a more revealing error
That's where I got the messages above (the screenshot). The other message I got from Application Insights.
Not relevant at all.
I did it all exactly the same way we did for the .net5 versions. I tried both upgrading the app and also deleting and creating a new one. Same results.
Tomorrow I'll try a simpler version of the app, with no logic, just to see if it works. If not, I'll try downgrading back to net5 for now.
Today I was able to find the problem. We use UserSecrets in our development to prevent from commiting database passwords and other settings to source control. We have a library in our internal Nuget feed that sets our defaults to Azure Functions at startup, and it add the UserSecrets as a configuration source.
The problem ended up being that something changed with AddUserSecrets. The code that worked in .net 5 had to be changed to include the parameter that defines UserSecrets as an optional configuration source. It was working on my local environment because I have a secrets.json file, but on Azure it failed because it couldn't find the file.
The difficulty in identifying the problem happened because the error message was not registered on Application Insights as an error, but as and information log. It should've been an error. The host process was not able to start the worker process and it logged the startup exception as information (actually, as multiple information logs, one for each line of the exception). After I figured that out and found the relevant error message, I was able to update our code to conform with this behavior change and the app now works.
Anyway, I see a problem here with how the startup failures are being reported to Insights (and not appearing at all in the Log Stream of the app). It should've been logged as an error and it should also have appeared in the Log Stream.
Looks like one place where dotnet-isolated functions still have a long road ahead in with regard to logs.
Looks like this was a change in AddUserSecrets
in .NET. See issue here. https://github.com/dotnet/runtime/issues/61418
@brettsam @gzuber Is it possible to log these exceptions as error and not information. And is there anything that needs to be done for this to be surfaced in the errors API? @soninaren
/cc @liliankasem who is looking into worker initialization errors
Yup I'm currently investigating issues that lead to a work init errors. I think this ties in to my design doc around improved logging, but also is slightly different in the sense that in this instance there was a reason why the worker process crashed.
@kelps do you mind sharing a timestamp, invocation id and region so I can look at the trace logs on our side, this should help me address the logging problem
@kelps do you mind sharing a timestamp, invocation id and region so I can look at the trace logs on our side, this should help me address the logging problem
@liliankasem Sure. I'll search my logs again. How do I share those with you? Is it ok to post that kind of data here?
@kelps do you mind sharing a timestamp, invocation id and region so I can look at the trace logs on our side, this should help me address the logging problem
@liliankasem Sure. I'll search my logs again. How do I share those with you? Is it ok to post that kind of data here?
Yup you're good to share that information here, nothing identifiable :)
Sorry for the delay, long meetings today....
timestamp of the first occurrency I found: 2021-12-09T13:26:43.8280758Z there is no invocation id in the logs for these errors and they are logged as Information, but you can filter cloud_RoleName == "REDCATED". the app and insights are running in a resource group in Brazil South. This is an app I'm using to test some concepts.
Edit: invocation id: 2a3a9490-5021-4b98-9a72-f7ed75b92e27
Found the invocation ID and updated your comment to remove the app name (region, invocation id and time is fine, but defo don't want to post app name 😄).
Looks like this was a change in
AddUserSecrets
in .NET. See issue here. dotnet/runtime#61418@brettsam @gzuber Is it possible to log these exceptions as error and not information. And is there anything that needs to be done for this to be surfaced in the errors API? @soninaren
/cc @liliankasem who is looking into worker initialization errors
Poke on this @soninaren, @brettsam @gzuber
I have a similar problem, but I don't use User Secres
Migrated from .net 5 functionapp v3 to .net 6 functionapp v4.
Works local, not remote on Azure (both Windows).
The message I get from the logs:
"2022-01-10T12:57:24.680 [Error] Failed to start a new language worker for runtime: dotnet-isolated.System.TimeoutException : The operation has timed out.at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()at async Microsoft.Azure.WebJobs.Script.Grpc.GrpcWorkerChannel.StartWorkerProcessAsync(CancellationToken cancellationToken) at D:\a\1\s\src\WebJobs.Script.Grpc\Channel\GrpcWorkerChannel.cs : 155at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??) at D:\a\1\s\src\WebJobs.Script\Workers\Rpc\FunctionRegistration\RpcFunctionInvocationDispatcher.cs : 118at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??) at D:\a\1\s\src\WebJobs.Script\Workers\Rpc\FunctionRegistration\RpcFunctionInvocationDispatcher.cs : 110at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.<>c__DisplayClass52_0.
Hello folks,
Is there any update?
I have faced a similar issue. The exception was logged. | property | Value |
---|---|---|
problemId | System.InvalidOperationException at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel | |
type | System.InvalidOperationException | |
assembly | Microsoft.Azure.WebJobs.Script, Version=4.0.0.0, Culture=neutral, PublicKeyToken=null | |
method | Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel | |
outerType | Microsoft.Azure.WebJobs.Host.FunctionInvocationException | |
outerMessage | Exception while executing function: Functions.get-orders | |
outerAssembly | System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e | |
outerMethod | System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw | |
innermostType | System.InvalidOperationException | |
innermostMessage | Did not find any initialized language workers | |
innermostAssembly | Microsoft.Azure.WebJobs.Script, Version=4.0.0.0, Culture=neutral, PublicKeyToken=null | |
innermostMethod | Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel |
This is still occuring and we have recreated the function app with dotnet-isolated and correct linuxFxVersion DOTNET-ISOLATED|6.0
The issue seems to be missing ASPNETCORE APP installation in the latest Linux image?
2022-02-14T20:56:50Z [Error] Final functionDispatcher state: WorkerProcessRestarting. Initialization timed out and host is shutting down 2022-02-14T20:56:50Z [Verbose] Sending invocation id:331cf876-5a1d-40c8-9c29-dfd7303f1ee1 2022-02-14T20:56:50Z [Verbose] Hosting stopping 2022-02-14T20:56:50Z [Verbose] Stopping file watchers. 2022-02-14T20:56:50Z [Verbose] Waiting for RpcFunctionInvocationDispatcher to shutdown 2022-02-14T20:56:50Z [Verbose] Draining invocations from language worker channel completed. Shutting down 'RpcFunctionInvocationDispatcher'
in my case (Function v4, VNet, Docker, Linux) - it helps to remove this line:
.ConfigureAppConfiguration(builder => builder.AddEnvironmentVariables())
in my case I was running host.RunAsync()
instead of host.Run()
in the Program.cs
file.
I had the same errors as above (Initialization timed out and host is shutting down) for an isolated function running on .NET 6 and Linux from a deployed zip. Id 0245e521-5cfa-443b-ba8d-b07a38b6b4d3
if it helps any debugging on the Microsoft side.
The fix for me was based on what @ds-evo mentioned. I remove the call to .ConfigureAppConfiguration()
when running in Azure.
#if DEBUG
.ConfigureAppConfiguration(x =>
{
x.AddJsonFile("local.settings.json");
x.AddJsonFile("host.json");
})
#endif
I had the same problems because of mixing .net 5 and .net 6 functionality, libraries and tooling. These are my lessons learned:
Az runtime
Create an azure function app with the right runtime (dotnet-isolated) and right version (4):
az functionapp create \
--resource-group *** \
--app-insights *** \
--consumption-plan-location westeurope \
--runtime dotnet-isolated \
--functions-version 4 \
--name *** \
--storage-account *** \
--os-type Linux
Program.cs
HostBuilder()
.ConfigureFunctionsWorkerDefaults()
.Build()
.Run();
Function attributes
Use the right Azure function attributes from the Microsoft.Azure.Functions.Worker
namespace!
[Function("TestFunction")]
public async Task<Response> Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = "v1/action")] HttpRequestData request)
{
var response = new Response()
{
Test = "test",
Data = request.CreateResponse()
};
response.Data.StatusCode = HttpStatusCode.Accepted;
return response;
}
internal class Response
{
public HttpResponseData Data { get; internal set; }
[QueueOutput("Queue", Connection = "AzureWebJobsStorage")]
public string Test { get; internal set; }
}
Deploy with the right tools
Deploy the function with azure-functions-core-tools.
npm i -g azure-functions-core-tools@4 --unsafe-perm true
mv local.settings.json.release local.settings.json
func azure functionapp publish $FUNCTION_APP
This is still occuring and we have recreated the function app with dotnet-isolated and correct linuxFxVersion DOTNET-ISOLATED|6.0
The issue seems to be missing ASPNETCORE APP installation in the latest Linux image?
@zyofeng Could you add this reference to the csproj
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>
and try it out again?
Facing the same issue for isolated, net6, v4, Function, windows
We tried to wrap up the call to .ConfigureAppConfiguration()
as suggested above.
We also have host.RunAsync()
But our call stack is a bit different from what has been posted above:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: [REDACTED]
---> System.InvalidOperationException: Did not find any initialized language workers
at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel(IEnumerable`1 rpcWorkerChannels) in /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcherLoadBalancer.cs:line 27
at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InvokeAsync(ScriptInvocationContext invocationContext) in /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs:line 389
at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 95
at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /_/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82
at Microsoft.Azure.WebJobs.Host.Executors.VoidTaskMethodInvoker`2.InvokeAsync(TReflected instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\VoidTaskMethodInvoker.cs:line 20
at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync(Object instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 52
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 596
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 542
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 320
--- End of inner exception stack trace ---
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 367
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 108
Facing the same issue for isolated, net6, v4, Function, windows
We tried to wrap up the call to
.ConfigureAppConfiguration()
as suggested above. We also havehost.RunAsync()
But our call stack is a bit different from what has been posted above:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: [REDACTED] ---> System.InvalidOperationException: Did not find any initialized language workers at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel(IEnumerable`1 rpcWorkerChannels) in /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcherLoadBalancer.cs:line 27 at Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InvokeAsync(ScriptInvocationContext invocationContext) in /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs:line 389 at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 95 at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /_/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82 at Microsoft.Azure.WebJobs.Host.Executors.VoidTaskMethodInvoker`2.InvokeAsync(TReflected instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\VoidTaskMethodInvoker.cs:line 20 at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync(Object instance, Object[] arguments) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 52 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 596 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 542 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 320 --- End of inner exception stack trace --- at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 367 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 108
I have the same issue using docker and Linux on Azure Portal
Running into this issue ("Did not find any initialized language workers") on one of our two .net 6 isolated functions (Linux Consumption Plan).
Figured out that the one failing (and Http Triggered function) was using Microsoft.Extensions.Azure.AddAzureClients
to add a ServiceBusClient for DI so we could send off service bus messages at completion, and the working function didn't have this in it's Program.cs.
Since it's an Http Triggered function, I couldn't find any way to set a Service Bus output binding as the returned result for Http trigger is already bound to the Http Response (am I correct on this?), so needed to go this approach instead.
Everything runs perfectly fine locally, but fails deployed to Azure.
So commented out the Microsoft.Extensions.Azure.AddAzureClients
call and our DI usages of the Service Bus Client, and no longer getting the error, it starts up and runs perfectly fine.
We have this exact same code being used in .Net 3.1 v3 Functions perfectly fine. I saw mentioned in one of the many issues logged about this very generic error that people had found that certain Nuget Packages aren't compatible with the Isolated Process functions. is Microsoft.Extensions.Azure
one of them? Is there a list or any way to know if a package won't be compatible? Or should this be working correctly and it's a bug?
Here's the code that causes the function to not startup. Tried both the Managed Identity and Connection String auth approaches thinking one might have an issue, but same result for both:
services.AddAzureClients(clientsBuilder =>
{
//clientsBuilder.AddServiceBusClientWithNamespace(serviceBusNamespace)
// .WithName(serviceBusClientName)
// .WithCredential(new DefaultAzureCredential(options));
clientsBuilder.AddServiceBusClient(configuration["TempSBConnectionString"])
.WithName(serviceBusClientName);
});
@willyt150 Just to confirm, did you have the TempSBConnectionString
(or any other settings value you have in your local environment) set up correctly in azure (Configuration->Application settings)?
@kshyju Yes, I double and triple checked to make sure that I wasn't going crazy, and all the settings I have in my local.settings.json
file exist up in the Azure Function's Application Settings.
In the case of TempSBConnectionString
I had actually created that up in the Azure function first and copied the value from there into my local.settings.json file
@kshyju I apologize for wasting your time. I have confirmed the issue was totally of my doing with the Managed Identity auth using the service bus namespace. Still not sure why it also failed w/ the Connection String approach though....
I'm reusing the "ServiceBusConnection__fullyQualifiedNamespace" app setting, since it's needed with the new Service Bus Trigger to support Managed Identity auth just keeping with the pattern, as the setting to retrieve the namespace from to pass into the AddServiceBusClientWithNamespace
call.
However, in the C#, I used the "__" instead of ":" for the configuration key, so on my Windows machine locally it worked just fine, but that caused the config value I retrieved up on the Linux machine to be an empty string.... so then the Add* call blew up.
Is there any way to expose those error messages during the DI process? Cause looking through app insights/log stream, I didn't see any of that information. Didn't figure this out until I wrote my own simple implementation of a service bus client factory, so when it hit my code and errored I had the ILogger available and got a very nice error telling me Value '' is not a valid Service Bus namespace.
@willyt150 Glad you got it figured out. We are working on improving the logging on worker startup failure scenario so that it will show detailed cause of failure in app-insights/log stream.
I've been trying to work around this issue for 4 days now without success. Until I stumbled across a SO comment about dependencies being on net5.0
. I migrated those manually to net6.0
and that resolved it for me.
I too am having this problem. This is an Azure Function .NET6 isolated mode.
The function works locally, works when published to an Azure Function, but does not work when published to a deployment slot (in either Consumption Mode or App Service Plan).
You may be thinking the deployment slot is missing a config/setting from the primary slot...I double-checked and I tested it. I published a different function that requires no configuration (the default Azure Function NET6 isolated template) and it too resulted in the error. The next test was to spin up a new function and add a deployment slot. The second deployment slot also produced the same error.
A conclusion is that NET6 isolated Azure Functions do not work with deployment slots.
The error is below:
2022-09-18T10:24:19.474 [Information] Host initialized (208ms)
2022-09-18T10:24:19.481 [Information] Host started (219ms)
2022-09-18T10:24:19.481 [Information] Job host started
2022-09-18T10:24:19.594 [Information] HttpOptions{"DynamicThrottlesEnabled": false,"EnableChunkedRequestBinding": false,"MaxConcurrentRequests": -1,"MaxOutstandingRequests": -1,"RoutePrefix": "api"}
2022-09-18T10:24:19.810 [Information] Stopping JobHost
2022-09-18T10:24:19.812 [Information] Job host stopped
2022-09-18T10:24:19.844 [Error] Failed to start a new language worker for runtime: dotnet-isolated.System.Threading.Tasks.TaskCanceledException : A task was canceled.
at async Microsoft.Azure.WebJobs.Script.Grpc.GrpcWorkerChannel.StartWorkerProcessAsync(CancellationToken cancellationToken)
at /_/src/WebJobs.Script.Grpc/Channel/GrpcWorkerChannel.cs : 159
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??)
at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 154
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??)
at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 146
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??)
at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 137
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.<>c__DisplayClass56_0.<StartWorkerProcesses>b__0(??)
at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 229
I too am having this problem. This is an Azure Function .NET6 isolated mode.
The function works locally, works when published to an Azure Function, but does not work when published to a deployment slot (in either Consumption Mode or App Service Plan).
You may be thinking the deployment slot is missing a config/setting from the primary slot...I double-checked and I tested it. I published a different function that requires no configuration (the default Azure Function NET6 isolated template) and it too resulted in the error. The next test was to spin up a new function and add a deployment slot. The second deployment slot also produced the same error.
A conclusion is that NET6 isolated Azure Functions do not work with deployment slots.
The error is below:
2022-09-18T10:24:19.474 [Information] Host initialized (208ms) 2022-09-18T10:24:19.481 [Information] Host started (219ms) 2022-09-18T10:24:19.481 [Information] Job host started 2022-09-18T10:24:19.594 [Information] HttpOptions{"DynamicThrottlesEnabled": false,"EnableChunkedRequestBinding": false,"MaxConcurrentRequests": -1,"MaxOutstandingRequests": -1,"RoutePrefix": "api"} 2022-09-18T10:24:19.810 [Information] Stopping JobHost 2022-09-18T10:24:19.812 [Information] Job host stopped 2022-09-18T10:24:19.844 [Error] Failed to start a new language worker for runtime: dotnet-isolated.System.Threading.Tasks.TaskCanceledException : A task was canceled. at async Microsoft.Azure.WebJobs.Script.Grpc.GrpcWorkerChannel.StartWorkerProcessAsync(CancellationToken cancellationToken) at /_/src/WebJobs.Script.Grpc/Channel/GrpcWorkerChannel.cs : 159 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??) at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 154 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??) at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 146 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.InitializeJobhostLanguageWorkerChannelAsync(??) at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 137 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcher.<>c__DisplayClass56_0.<StartWorkerProcesses>b__0(??) at /_/src/WebJobs.Script/Workers/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs : 229
I resolved my issue...The function name was too long. The default name provided by Visual Studio when publishing makes the name too long for Azure as described here: https://github.com/Azure/azure-functions-host/issues/2015
Create an Application Setting with the key of "AzureFunctionsWebHost:hostid" then give the host id a unique name less than 32 characters. Make it a deployment slot setting.
I had this issue a year ago when running on .NET 5 and v3 Functions, upgraded to .NET 6 and v4, thinking this surely has to be fixed by now and clearly it isnt. I applied this work around again (as I did originally) and it's working again.
https://github.com/Azure/azure-functions-dotnet-worker/issues/524#issuecomment-872388417
I just update the project.csproj with following
`
I just update the project.csproj with following
<PublishReadyToRun>true</PublishReadyToRun> <RuntimeIdentifier>win-x86</RuntimeIdentifier>
After deploying and restart again I see functions are available
Can checkout the following docs
Another solution is to publish in x64 and set the platform to 64-bit
We have a few Azure Functions developed using isolated-process in .net 5 and we are migrating to .net 6.
The migration worked without problems and runs perfectly local, but fails when deployed to azure. We are using Http, ServiceBus and Timer triggers and all fail the same way:
It shows an "Executing" log in the "Log stream", but after a few seconds it fails with "[Error] Final functionDispatcher state: WorkerProcessRestarting. Initialization timed out and host is shutting down". After that the host is restarted.
All functions fail the same way, but only when running on Azure. They work perfectly local and there are no other logs on Azure to find out what could be going on.
The app was not created from scratch, but was updated from the previous version to use .net 6 and updating/upgrading all required dependencies.
Edit:
Application Insights shows the following 2 error log messages, aprox. 50 seconds after any attempts: Final functionDispatcher state: WorkerProcessRestarting. Initialization timed out and host is shutting down and Did not find any initialized language workers