Support function to modify import paths

[Timmmm]

I'm working on a codebase that does some unfortunate magic manipulation of import paths, e.g. it converts

import proj.xm_cpu.major.xm_cpu_pkg

to

import proj.xm_cpu.mk1.lib.python.xm_cpu_pkg

It's done by adding a custom module finder to sys.meta_path in sitecustomize.py.

Pyright quite reasonably can't handle this. The only semi-reasonable way I can think to fix it is to allow passing a Javascript function to Pyright that it passes import paths through before trying to load them. Is that possible?

[erictraut]

That form of extensibility isn't something we would ever provide. However, there are other options.

You could use a stub that provides the necessary static redirection. The exact details depend on whether the imported project is part of your workspace (e.g. in a monorepo) or is a library that lives outside of your workspace and is installed as a dependency. If it's the former, then you can add the type stub file in place (i.e. name it proj/xm_cpu/major/xm_cpu_pkg/__init__.pyi). If the library code lives outside of your workspace, you can create the stub within a workspace-local stub path. By default, the stub path is called "typings", but you can specify whatever subdirectory name you would like by specifying the "stubPath" setting. Place the type stub file there (e.g. typings/xm_cpu/major/xm_cpu_pkg/__init__.pyi). That type stub would then import and re-export the symbols from the other location.

from proj.xm_cpu.mk1.lib.python.xm_cpu_pkg import *

[Timmmm]

Aha, that worked almost perfectly! It gives a new lint/warning:

Import "proj.xm_cpu.major.xm_cpu_pkg" could not be resolved from source Pylance(reportMissingModuleSource)

But everything seems to work anyway - code completion, go to definition etc. Maybe I'll just turn that warning off.

Not a completely zero effort solution since this project has gazillion of those directories but I can write a script to generate the __init__.py files. And I totally understand why you wouldn't want to add this option. I don't think I would either to be honest.

Thanks for the quick reply!

[Timmmm]

Actually I spoke too soon - it doesn't quite work. It seems to work if you have this:

import proj.xm_cpu.major.xm_cpu_pkg

And you can ctrl-click it and it goes to the __init__.pyi file. But if you have this:

from proj.xm_cpu.major.xm_cpu_pkg import SomeType

Then it says SomeType is unknown import symbol even though you can ctrl-click xm_cpu_pkg to go to __init__.pyi, and from there you can ctrl-click on xm_cpu_pkg again to go to the xm_cpu_pkg.py which contains SomeType.

:-/ I might just give up.

[erictraut]

It sounds like you're close to getting it working.

You need to make sure that SomeType is re-exported from the __init__.pyi. By default, symbols imported by a stub file are not re-exported from that stub. A re-export is specified through the use of a wildcard import or a redundant import form like from .x import SomeType as SomeType. For details, refer to PEP 484 or this documentation.

[Timmmm]

Sorry I might be being stupid but doesn't the bit that says If a wildcard import (of the form “from X import *”) is used, all symbols referenced by the wildcard are not private. mean that the symbols should be exported?

My __init__.pyi stub file looks like this:

from proj.xm_cpu.mk1.lib.python.xm_cpu_pkg import *

So I would have thought everything in xm_cpu_pkg should be re-exported?

Thanks for the help!

[Timmmm]

Wait, cancel that! I got it to work. The issue was that I also had a file at typings/proj/xm_cpu/mk1/lib/python/xm_cpu_pkg/__init__.pyi which is the same path (plus typings/) as the actual correct import, so I guess that was confusing it.

Without that file it all works, thanks!