pyright does not respect pyproject.toml defined extraPaths

[Stewart86]

As mentioned in #10. Opening a new ticket here to keep track of this issue.

Context

For some specific use case where extraPaths is defined in the pyproject.toml (example below). After activating VenvSelect venv, pyright no longer respect the extra paths defined.

pyproject.toml

...

[tool.pyright]
executionenvironments = [
  { root = "", extrapaths = ["layer/"]},
  { root = "my_serverless_app/lambda_functions/", extrapaths = ["layer/"]},
]

...

Investigation

Even when extraPaths is declared in the lspconfig as such.

lspconfig.lua

return function(opts)
  opts.settings = {
    python = {
      analysis = {
        ...
        -- both relative and absolute path doesn't work
        extraPaths = { "layer/", "<absolute>/<path>/layer/" },
      },
    },
  }
  return opts
end

However, when extraPaths is added into venv.lua

venv.lua

-- Hook into lspconfig so we can set the python to use.
M.set_pythonpath = function(python_path)
    lspconfig.pyright.setup({
        before_init = function(_, c)
            c.settings.python.pythonPath = python_path
            c.settings.python.analysis.extraPaths = { "<absolute>/<path>/layer/" }
        end,
    })
end

Also, before activating VenvSelect pyright knows my extra paths but is not aware of the venv paths. Which was reflected with errors to the externally imported packages. As soon as venv is activated, the import errors switched from showing errors to my internal packages and nothing for the external packages.

To Replicate

This particular use case is mainly for building a AWS serverless application where Lambda Functions and Layer live in a separate runtime environment but stay in a single repository. Of cause there might be other use cases which I am not aware of. Directory structure looks similar to this.

directory tree

my_serverless_app/
    layer/
        utils.py
        requirements.txt
    lambda_functions/
        function_one/
            main.py
            requirements.txt
pyproject.toml
.git

In normal circumstances main.py in function_one/ folder should import utils.py as such

my_serverless_app/lambda_functions/function_one/main.py

from my_serverless_app.layer import utils

But this will break on AWS Lambda runtime environment because the root directory is not being included into the runtime. So we can only import as such

my_serverless_app/lambda_functions/function_one/main.py

import utils

That means, we need to declare extraPaths in pyproject.toml for pyright to know that we are directly importing from a particular folder.

[Stewart86]

I don't know if there is a better way, but I am thinking if we parse pyproject.toml for the extra paths and append it into venv.lua that would work. however, I feel it's kind of hackish.

[linux-cultist]

Yeah I agree. This week I'm starting a new job so won't have much free time, but if you have time and energy, please think about a good solution to this. I really like that you have some ideas already and maybe the hackish solution can be OK if we don't find another one for now....

See what you can come up with (if you have time of course).

[Stewart86]

Now that my PR #12 to resolve this issue have been merged into main. This issue is being resolved. Closing this for now!