indygreg / PyOxidizer

A modern Python application packaging and distribution tool
Mozilla Public License 2.0
5.4k stars 234 forks source link

`importlib.import_module` raising ModuleNotFoundError with custom sys.path #664

Open pquentin opened 1 year ago

pquentin commented 1 year ago

Hello! I'm porting Rally (the official Elasticsearch benchmark tool) to PyOxidizer, and despite being stuck on 0.22.0 it works really well and required almost no change to our code. However Rally has this feature where it will load custom Python modules from another directory, for example https://github.com/elastic/rally-tracks/blob/master/eql/track.py.

Suppose /home/q/src/pyapp/other/path/amodule.py exists. The following CPython code does not raise an exception and loads the module:

import importlib
import sys

sys.path.insert(0, "/home/q/src/pyapp/other/path")
importlib.import_module("amodule")

But translated to PyOxidizer, I get an exception. Here's the pyoxidizer.bzl file that attempts to load the same module:

def make_exe():
    dist = default_python_distribution()
    policy = dist.make_python_packaging_policy()
    policy.resources_location = "in-memory"

    python_config = dist.make_python_interpreter_config()
    # Evaluate a string as Python code when the interpreter starts.
    python_config.run_command = """
import importlib
import sys

sys.path.insert(0, "/home/q/src/pyapp/other/path")
importlib.import_module("amodule")
"""

    return dist.to_python_executable(
        name="pyapp",
        packaging_policy=policy,
        config=python_config
    )

def make_install(exe):
    files = FileManifest()
    files.add_python_resource(".", exe)
    return files

register_target("exe", make_exe)
register_target("install", make_install, depends=["exe"], default=True)

resolve_targets()

But pyoxidizer run fails with:

installing files to /home/q/src/pyapp/./build/x86_64-unknown-linux-gnu/debug/install
Traceback (most recent call last):
  File "<string>", line 4, in <module>
  File "importlib", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'amodule'
error: cargo run failed

I would appreciate any ideas to help me debug this, or even turns this into a reproducer using only oxidizer_importer.