search

projectkudu / kudu

Kudu is the engine behind git/hg deployments, WebJobs, and various other features in Azure Web Sites. It can also run outside of Azure.
Apache License 2.0
3.12k stars 652 forks source link

Job failed due to exit code -1073740791 #3015

Closed kimpenhaus closed 6 months ago

kimpenhaus commented 5 years ago

Hi all,

we do have several continuous WebJobs (14 in number) running in an AppService on Azure (Sub-ID is c7f0...edc3). The WebJobs are deployed by copy deployment through a release pipeline on Azure DevOps.

The WebJobs are .NET core 2.1 console applications, connecting to an azure service bus and processing messages. In general they are doing its job for a couple of hours. But from time to time the are failing ungraceful with exit code -1073740791 leaving the currently processed messages in an uncertain state. The amount of errors vary from job to job - but they all face it. One of the jobs e.g. failed 7 times in the last 24h)

Logging is done by AppInsights Telemetry.

The WebJobs are not build by the Azure WebJobs SDK although they are using the WebJobsShutdownWatcher to handle graceful shutdowns, restarts etc. Also we do have a special "handler" for unhandled appdomain-exceptions, to push/flush the client cached loggs to AppInsights (as mentioned in other tickets we've found).

None of these ungraceful shutdowns could be seen in the AppInsights logs (which we think is quite usual)

What we can see in the Kudu WebJobs logs is as follows:

[08/14/2019 02:43:05 > 966921: SYS ERR ] Job failed due to exit code -1073740791
[08/14/2019 02:43:05 > 966921: SYS INFO] Process went down, waiting for 0 seconds
[08/14/2019 02:43:05 > 966921: SYS INFO] Status changed to PendingRestart
[08/14/2019 02:43:08 > 966921: SYS INFO] Run script 'run.cmd' with script host - 'WindowsScriptHost'
[08/14/2019 02:43:08 > 966921: SYS INFO] Status changed to Running

We tried to check the error code here: https://errorcodelookup.com/?type=ntstatus&code=C0000409 which doesn't help very much.

We`ve read in case of other failure codes that the dashboard logging is deprecated on should be turned off, we were not able to do so (might be the fact we are not using the sdk) but to be honest we do not know if our exit codes depends on the same issues why the dashboard loggin should be turned off. We've tried to lower the maxConcurrentCalls as well as the prefetchCount - none of these helped.

We would appreciate any help leading to a solution to this problem as well as any suggestions for improvements. If any further information is needed we thankfully will provide!

Thanks, Marcus

jvano commented 6 months ago

Hi

If the problem persists and is related to running it on Azure App Service, please open a support incident in Azure: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request

This way we can better track and assist you on this case

Thanks,

Joaquin Vano Azure App Service