pyoxidizer 0.17 init-rust-project builds but fails with cargo.

[julienfr112]

On ubuntu 20.04 :

pyoxidizer --version
PyOxidizer 0.17.0
commit: unknown
source: https://github.com/indygreg/PyOxidizer.git
pyembed crate location: version = "0.17.0"
~/test/testpyo$ pyoxidizer init-rust-project testpyo

works ok with pyoxidizer run. but fails with cargo run

~/test/testpyo$ cargo run --features allocator-jemalloc
   Compiling fs_extra v1.2.0
   Compiling pyembed v0.17.0
   Compiling testpyo v0.1.0 (/home/julien/test/testpyo)
   Compiling jemalloc-sys v0.3.2
    Finished dev [unoptimized + debuginfo] target(s) in 58.66s
     Running `target/debug/testpyo`
SystemError: null argument to internal routine
Fatal Python error: init_importlib_external: external importer setup failed
Python runtime state: core initialized

Current thread 0x00007fcaa90c4000 (most recent call first):
<no Python frame>

Reading the doc, I discovered that setting the same python that the one fetched by pyoxidizer could help :

cargo clean
$ PYTHON_SYS_EXECUTABLE=/home/julien/test/testpyo/python/python/install/bin/python3 cargo run --features allocator-jemalloc
   Compiling python3-sys v0.6.0
   Compiling pyembed v0.17.0
   Compiling cpython v0.6.0
   Compiling testpyo v0.1.0 (/home/julien/test/testpyo)
    Finished dev [unoptimized + debuginfo] target(s) in 11.48s
     Running `target/debug/testpyo`
Python path configuration:
  PYTHONHOME = '/home/julien/test/testpyo/target/debug'
  PYTHONPATH = (not set)
  program name = '/home/julien/test/testpyo/target/debug/testpyo'
  isolated = 1
  environment = 0
  user site = 0
  import site = 1
  sys._base_executable = '/home/julien/test/testpyo/target/debug/testpyo'
  sys.base_prefix = '/home/julien/test/testpyo/target/debug'
  sys.base_exec_prefix = '/home/julien/test/testpyo/target/debug'
  sys.platlibdir = 'lib'
  sys.executable = '/home/julien/test/testpyo/target/debug/testpyo'
  sys.prefix = '/home/julien/test/testpyo/target/debug'
  sys.exec_prefix = '/home/julien/test/testpyo/target/debug'
  sys.path = [
    '/home/julien/test/testpyo/target/debug/lib/python39.zip',
    '/home/julien/test/testpyo/target/debug/lib/python3.9',
    '/home/julien/test/testpyo/target/debug/lib/python3.9/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fbf4f445000 (most recent call first):
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 763 in create_module
  File "<frozen importlib._bootstrap>", line 565 in module_from_spec
  File "<frozen importlib._bootstrap>", line 666 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load

but it fails with another error.

Idem on Mac Os ( mac mini A1), but with the encoding error

I'm really not sure if it should work or not, but I find it the only way to have a rust project that embed a python interpreter. As everything is quite complicated ( pyembed is calling pyoxidier, or the revese, or both ... ) , and the doc is still work in progress, I think the init-rust-project should work out of box with cargo.

On top of that, pyoxidier run does not seems to care at all about the src/main.rs, and just lauching a python exe.

[julienfr112]

I solved the problem, but not easy at all, and mostly undocumented. (I had to look at pyoxidizer src ...)

=> add "cpython-link-default" to default feature ( not documented ) => add "allocator-jemalloc" to default feature ( was in the error message) => add PYTHON_SYS_EXECUTABLE=... as documented in https://pyoxidizer.readthedocs.io/en/latest/pyoxidizer_rust_cargo_source_checkouts.html

I can make an Pull requests witch customize init-rust-project's cargo.toml with

default = ["build-mode-pyoxidizer-exe", "allocator-jemalloc", "cpython-link-default"]

would you be interested in merging this Pull Requests ?

[hamzamohdzubair]

I discovered the same thing, pyoxidizer run does not even read src/main.rs and local buld.rs. The only way that it seems we can embed python in our rust project is by building with cargo. I filed this issue #439 in relation to that problem.

I was getting the exact error you got, and thanks to your help, by adding in default in Cargo.toml i was able to get rid of that error. But now i am getting different error: It's unable to find my python module: test.py in project root

No module named test

[hamzamohdzubair]

UPDATE:

Solved it. Had to input the correct path in pyoxidizer.bzl in

exe.add_python_resources(
    exe.read_package_root(
        path=<fullpath>,
        packages=["test"]))

Thanks @julienfr112.