In pypa/pip#11812 I described a situation where a leaky environment can lead to a BackendInvalid exception, even when the backend is valid and defines a backend-path that contains the correct code to be imported.
This can happen when a MetaPathFinder preceeds importlib.machinery.PathFinder in sys.meta_path, and is very difficult to debug, because there is no obvious problem in the backend (at least to my best knowledge)…
My understanding is that is part of the responsibilities of the frontend to use the tools available to load the backend from the specified backend-path, based on the following excerpt of PEP 517:
The backend code MUST be loaded from one of the directories specified in backend-path (i.e., it is not permitted to specify backend-path and not have in-tree backend code).
Note if the backend provides a correct specification for backend-path with the corresponding in-tree files, but the frontend loads it from a different location, the PEP 517’s restriction would still not be satisfied.
My opinion is that a better approach would be to use importlib (e.g. importlib.util.spec_from_file_location or importlib.machinery.PathFinder.find_spec) to actively search only the specified paths and ignore other meta path finders.
Please note that PEP 517 does not seem to prohibit situations where sys.meta_path is manipulated. In fact, the text even mentions a few situations (including virtual environment creation with --system-site-packages), which could lead to such circumstances:
A build frontend MAY use any mechanism for setting up a build environment that meets the above criteria. For example, simply installing all build-requirements into the global environment would be sufficient to build any compliant package – but this would be sub-optimal for a number of reasons.
… a build frontend could have a --build-with-system-site-packages option that causes the--system-site-packages option to be passed to virtualenv-or-equivalent when creating build environments (…)
Even if pip improves environment isolation, the PEP does not rule out the possibility of keeping customizations in sys.meta_path. Therefore, inserting entries to sys.path and expect importlib.import_module to work may be subject to errors.
In pypa/pip#11812 I described a situation where a leaky environment can lead to a
BackendInvalidexception, even when the backend is valid and defines abackend-paththat contains the correct code to be imported.This can happen when a
MetaPathFinderpreceedsimportlib.machinery.PathFinderinsys.meta_path, and is very difficult to debug, because there is no obvious problem in the backend (at least to my best knowledge)…My understanding is that is part of the responsibilities of the frontend to use the tools available to load the backend from the specified
backend-path, based on the following excerpt of PEP 517:Note if the backend provides a correct specification for
backend-pathwith the corresponding in-tree files, but the frontend loads it from a different location, the PEP 517’s restriction would still not be satisfied.My opinion is that a better approach would be to use
importlib(e.g.importlib.util.spec_from_file_locationorimportlib.machinery.PathFinder.find_spec) to actively search only the specified paths and ignore other meta path finders.Please note that PEP 517 does not seem to prohibit situations where
sys.meta_pathis manipulated. In fact, the text even mentions a few situations (including virtual environment creation with--system-site-packages), which could lead to such circumstances:Even if
pipimproves environment isolation, the PEP does not rule out the possibility of keeping customizations insys.meta_path. Therefore, inserting entries tosys.pathand expectimportlib.import_moduleto work may be subject to errors.