"func host start" ignore venv and uses global python interpreter

[fanshaoshan]

When developing azure function using python, any new packages added to requirements.txt cannot be found when debugging locally. It will always cause "Module not found" error, deploy to cloud has no issue.

I did some tests, and it turns out that even in a venv, but "func host start" will still use the global interpreter

Here are the steps to reproduce this. Using git-bash, same result when using powershell, before run function, I edit init.py to print out sys.path and sys.executable to check which python been used.

[🤖💤] ~/Workspace/azfunc2 $ python -m venv .venv

[🤖💤] ~/Workspace/azfunc2 $ source .venv/Scripts/activate

(.venv) [🤖💤] ~/Workspace/azfunc2 $ func init Use the up/down arrow keys to select a worker runtime:python Found Python version 3.7.9 (python3). Writing requirements.txt Writing .funcignore Writing getting_started.md Writing .gitignore Writing host.json Writing local.settings.json Writing C:\Users\shfan\Workspace\azfunc2.vscode\extensions.json

(.venv) [🤖💤] ~/Workspace/azfunc2 $ func new HttpTrigger Use the up/down arrow keys to select a template:HTTP trigger Function name: [HttpTrigger] Writing C:\Users\shfan\Workspace\azfunc2\HttpTrigger__init__.py Writing C:\Users\shfan\Workspace\azfunc2\HttpTrigger\function.json The function "HttpTrigger" was created successfully from the "HTTP trigger" template.

(.venv) [🤖💤] ~/Workspace/azfunc2 $ vim HttpTrigger/init.py

(.venv) [🤖💤] ~/Workspace/azfunc2 $ func host start Found Python version 3.7.9 (python3).

Azure Functions Core Tools Core Tools Version: 4.0.4483 Commit hash: N/A (64-bit) Function Runtime Version: 4.1.3.17473

Functions:

    HttpTrigger: [GET,POST] http://localhost:7071/api/HttpTrigger

For detailed output, run func with --verbose flag. [2022-04-29T22:25:44.326Z] Worker process started and initialized. [2022-04-29T22:25:47.685Z] Executing 'Functions.HttpTrigger' (Reason='This function was programmatically called via the host APIs.', Id=2c032d7d-5dcc-40fc-88f7-bbc8f211e7c8) [2022-04-29T22:25:47.744Z] Python HTTP trigger function processed a request. [2022-04-29T22:25:47.748Z] ['C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.7\WINDOWS\X64', 'C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.7\WINDOWS\X64', 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64qbz5n2kfra8p0\python37.zip', 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64qbz5n2kfra8p0\DLLs', 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64qbz5n2kfra8p0\lib', 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64qbz5n2kfra8p0', 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\site-packages', 'C:\Users\shfan\Workspace\azfunc2'] [2022-04-29T22:25:47.749Z] C:\Users\shfan\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\python.exe [2022-04-29T22:25:47.805Z] Executed 'Functions.HttpTrigger' (Succeeded, Id=2c032d7d-5dcc-40fc-88f7-bbc8f211e7c8, Duration=138ms)

This is on my Windows 11 laptop, tested with python 3.7, 3.8, 3.9, all have the same issue, but on another windows 10 with python 3.7, works fine, not sure if this is related to OS or some settings.

[fanshaoshan]

Seems this is related to how python has been installed.

When install python from MS Store, if always has issue, but I tried to download python from python.org and install from MSI, issue gone.

[apawast]

@gavin-aguiar @vrdmr could you please help investigate this?

[trwhitcomb]

I'm thankful to see I wasn't the only one! I have also encountered this issue with the same debugging steps. Removing the Python from the MS store and installing Python from python.org MSI seems to allow the proper venv Python to be used to execute the func host start command for me as well. In my case, Python version was 3.9.13

[friediisch]

Hi! I am having the same issue on an M1 MBA. Downloading and installing Python 3.8.10 sets the correct global interpreter but func start still uses Python 3.6.6 and throws a "Failed to initialize worker provider for: /usr/local/Cellar/azure-functions-core-tools@4/4.0.4544/workers/python". Any ideas?

[thec0dewriter]

Hello, i have a similar issue however just as an addition: In case i'm running the function throgh cmd with core tools the environment seems to be picked up properly

 (.venv) (durable_functions) D:\workspace\functions>func host start
Found Python version 3.8.8 (py).

Azure Functions Core Tools
Core Tools Version:       4.0.4829 Commit hash: N/A  (64-bit)
Function Runtime Version: 4.11.2.19273

But when I run through vscode with debug the environment seems to be an old one:

Executing task: func host start 

Found Python version 3.7.3 (py).

Azure Functions Core Tools
Core Tools Version:       4.0.4829 Commit hash: N/A  (64-bit)
Function Runtime Version: 4.11.2.19273

[ArashJavan]

Same issue here also!

[mhwaage]

Same issue, also on windows. The relevant code appears to be here.

When finding python, func looks for and evaluates python versions in the following order:

1. According to an environment variable languageWorkers:python:defaultExecutablePath
2. Looking for py (the windows python launcher)
3. Looking for python3
4. Looking for python
5. -- 9. Looking for python3.6-python3.10

It then chooses the first of those that is a supported python version.

On windows, virtual environemnts only contain the python executable, so if they are in path, py and python3 override whatever is in the virtual environment. In my case, py points to python 3.11, which is not supported so it is skipped, then python3 points to a python 3.8 installed via the windows store and is chosen.

The issue can be resolved by altering the search order - if func first chooses the python executable, it will correctly pick the venv version of python on both windows and unix. I.e., changing lines 225-235 to instead read

  var versions = new List<WorkerLanguageVersionInfo>
  {
      await pythonGetVersionTask,
      await pyGetVersionTask,
      await python3GetVersionTask,
      await python36GetVersionTask,
      await python37GetVersionTask,
      await python38GetVersionTask,
      await python39GetVersionTask,
      await python310GetVersionTask,
  };

Regarding the environment variable, it seems to be intended for users who have a complicated environment, i.e. as a workaround for precisely these issues. Is it documented somewhere that it can be used?

[mhwaage]

It would be helpful if func could be configured to only use the first python it comes across, to avoid developers developing within a virtual environment with an unsupported python version erroneoussly thinking that their choice works.

[perrin7]

I have this problem too, with the issue being that the initial commands for the pip install run in the .venv but the func host start then runs in the global python interpreter.

It's incredibly frustrating and I can't find a fix, except for installing all the pip packages in the global interpreter as well, but it defeats the purpose of the venv!

[mhwaage]

@perrin7 You can probably work around it by copying the virtual-environment python into another file called py.exe next to it, e.g. copy-item .venv\Scripts\python.exe .venv\Scripts\py.exe. Alternatively, make a symlink, just note that if you do that via explorer, then explorer will hide a .lnk suffix even if you have explorer configured to show suffixes, so you will have to rename it in a terminal.

[perrin7]

@mhwaage - Good idea! I tried, but the func host start still goes direct to the global 'py.exe' rather than the one in .venv\Scripts\py.exe 👎

[mhwaage]

Oh that's interesting! I see now from the issue log that py is supposed to auto-detect that it is in a virtual environment and launch the correct python based on that, which seems to fail - different from what I had expected!

[tim-steinkuhler]

I'm on a Mac and was running into the same issue, can get func start to use the correct python executable by setting the environment variable mentioned above, before running func start, by using this command: env languageWorkers:python:defaultExecutablePath=.venv/bin/python func start

Inspiration found here: https://stackoverflow.com/questions/47827887/bash-zsh-export-environment-variable-with-name-containing-a-colon

[princeanire]

Hey, Just wondering if this issue has any progress, as of April 15, 2023 the problem still exists,

I'm using Windows 11 and Visual Studio Code, python 3.10 installed through Microsoft Store, my Azure Functions is created through Visual Studio Code command palette, it even show me a dialog which python interpreter to use, but when I start running func host it throws out module not found, I fixed it by installing it on the global interpreter but this Is not it should be solve

[FWin22]

same issue here. it just ignores the venv

[satadru5]

@perrin7 You can probably work around it by copying the virtual-environment python into another file called py.exe next to it, e.g. copy-item .venv\Scripts\python.exe .venv\Scripts\py.exe. Alternatively, make a symlink, just note that if you do that via explorer, then explorer will hide a .lnk suffix even if you have explorer configured to show suffixes, so you will have to rename it in a terminal.

It worked.. Great idea. But without any link

[MaheshSQL]

[Windows] Hi team. This issue is a roadblock for my work. The Azure Function Core Tools is ignoring the conda virtual environment from which I initialise func start command. I tested uninstalling the latest version and falling back on old versions, no luck.

---

UPDATE

Seems to use the virtual environment after uninstalling the default Python from Program Files.

[ncostar]

Adding "azureFunctions.pythonVenv": "<path to your venv>", to your .vscode\settings.json file solved this issue for me.

[hbcondo]

Adding "azureFunctions.pythonVenv": "<path to your venv>", to your .vscode\settings.json file solved this issue for me.

Thank you @ncostar, this worked for me too: "azureFunctions.pythonVenv": "\\.venv"

[golfalot]

I'm on a Mac and was running into the same issue, can get func start to use the correct python executable by setting the environment variable mentioned above, before running func start, by using this command: env languageWorkers:python:defaultExecutablePath=.venv/bin/python func start

Inspiration found here: https://stackoverflow.com/questions/47827887/bash-zsh-export-environment-variable-with-name-containing-a-colon

I propose that this could be high tariff when deploying to production and the runtime is not found in that path ?

[deanm0000]

I just bumped into this issue on WSL ubuntu and realized that my venv was python 3.12 whereas AZ functions only supports up to 3.11 so I deleted it and reformed from 3.11 and then it was ok after that.

[gabrielSoudry]

Same problem as @deanm0000, juste remove my venv in python3.12 and use venv in python 3.11 work for me :)

[leonavevor]

fixed it by adding "azureFunctions.pythonVenv": " ${workspaceFolder}/.venv" in .vscode\settings.json

this how my config looks:

in settings.json >>>

{
    "azureFunctions.deploySubpath": "code\\backend\\batch",
    "azureFunctions.scmDoBuildDuringDeployment": true,
    "azureFunctions.pythonVenv": "${workspaceFolder}/.venv",
    "azureFunctions.projectLanguage": "Python",
    "azureFunctions.projectRuntime": "~4",
    "debug.internalConsoleOptions": "neverOpen",
    "azureFunctions.projectLanguageModel": 2,
    "githubPullRequests.ignoredPullRequestBranches": [
        "main"
    ],
    "python.testing.pytestArgs": [
        "."
    ],
    "python.testing.cwd": "${workspaceFolder}/code",
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
}

in launch.json >>>

{
    "version": "0.2.0",
    "configurations": [

        {
            "name": "Attach to Python Functions",
            "type": "python",
            "request": "attach",
            "port": 9091,
            "preLaunchTask": "func: host start"
        },
...

in task.json >>>

{
    "version": "2.0.0",
    "tasks": [
        {
            "type": "func",
            "label": "func: host start",
            "command": "host start",
            "problemMatcher": "$func-python-watch",
            "isBackground": true,
            "dependsOn": "pip install (functions)",
            "options": {
                "cwd": "${workspaceFolder}/code/backend/batch"
            }
        },
        {
            "label": "pip install (functions)",
            "type": "shell",
            "osx": {
                "command": "${config:azureFunctions.pythonVenv}/bin/python -m pip install -r requirements.txt"
            },
            "windows": {
                "command": "${config:azureFunctions.pythonVenv}\\Scripts\\python -m pip install -r requirements.txt"
            },
            "linux": {
                "command": "python -m pip install -r ${workspaceFolder}/code/backend/requirements.txt || true"
            },
            "problemMatcher": [],
            "options": {
                "cwd": "${workspaceFolder}/code/backend"
            }
        },
....

[konstantinmiller]

If I run func start from the command line and not from VSCode, which environment variable do I need to set, as the equivalent to azureFunctions.pythonVenv?

EDIT: Issue solved. I had Python 3.12 in my venv and then func silently picked a different Python from the system.