Error: Did not find any initialized language workers - Time triggered python function, docker

[aleksandraloka]

Our Azure Function, deployed on Dedicated plan (AppServicePlan) experienced a sudden error: Did not find any initialized language workers. Previously, it was working without any issue. It's a simple TimerTriggered function. We only redeployed the app with a simple change of the scheduled time, and the problems started. It's a dockerized python function.

What we tried? We changed the FUNCTIONS_EXTENSION_VERSION to ~4. Previously it was ~3. We added the FUNCTIONS_WORKER_RUNTIME to python. Previously, we did not add this because it is a dockerized app. We created new appServicePlan only for these azureFunctions. Previously, other appSerivces were part of the same Plan. Also, we recreated the FunctionApp via CDKTF code.

So, after doing these changes, the function starts working for few executions, but after some time it generates again the same error. Our main function app has several TimerTriggered functions (several init.py functions) as part of the AzureFunctions. Some of them may even have same schedule time. Therefore we added the following env var, too, thinking it could help with the execution of the several functions: FUNCTIONS_WORKER_PROCESS_COUNT=4

Also, the same functions are working all right on an environment (different tenant, same configs) that was not updated (the function was not redeployed there). And, what seems to be working for one function on one tenant, does not work on the other. It's an inconsistent behaviour.

We even tried changing the init.py with a generic Timer code, but the problem persisted.

def main(mytimer: func.TimerRequest) -> None:
    utc_timestamp = datetime.datetime.utcnow().replace(
        tzinfo=datetime.timezone.utc).isoformat()

    if mytimer.past_due:
        logging.info('The timer is past due!')

More configuration

host.json file:

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[2.*, 3.0.0)"
  }
}

Dockerfile:

FROM mcr.microsoft.com/azure-functions/python:4-python3.9-appservice

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
    AzureFunctionsJobHost__Logging__Console__IsEnabled=true

COPY requirements.txt /
RUN pip install -r /requirements.txt

COPY . /home/site/wwwroot

Investigative information

• Timestamp: 2023-06-14T23:09:00.0034239+00:00
• Function App name: verisdev-expired-tokens-monitor-eastus-azure-function
• Function name(s) (as appropriate):
• Core Tools version:
• execution Id: Id=624351c2-b5ab-41e5-8c97-b38f7a138771
• region: eastUs

Repro steps

The problem started with redeploying the app with different scheduled time: example: "schedule": "0 /1 *"

So it's not a configuration/dependency change of any kind.

The following generic error occured: Did not find any initialized language workers

Exception type - System.InvalidOperationException

Failed method - Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcFunctionInvocationDispatcherLoadBalancer.GetLanguageWorkerChannel

Contents of the requirements.txt file:

Provide the requirements.txt file to help us find out module related issues.

requirements.txt:

azure-functions
requests==2.28.1

[aleksandraloka]

update: this seems to be an issue in EastUs region. We have tried the same functions in other region, and it worked ok.

As a further note, we observed that the host version in eastus2 that has no problems has the following version: "version": "4.21.1.1", "versionDetails": "4.21.1+85b2144d318ea423fb58c2a07f001dd81750d267", "platformVersion": "100.0.7.287"

However, the eastus host version is 4.21.1.0.

[bhagyshricompany]

Thanks for reporting the issue. Will check and update you.

[bhagyshricompany]

@aleksandraloka I check same in east us region after deployed working fine as per your repro steps.Can you try once again.

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.