scientific-python / lazy-loader

Populate library namespace without incurring immediate import costs
BSD 3-Clause "New" or "Revised" License
118 stars 19 forks source link

Allow to not fail on stub attach in frozen apps #38

Closed Czaki closed 1 year ago

Czaki commented 1 year ago

I have set up a pre-release test for pyinstaller freeze of my application.

I just found that the import of scikit-image rc fail because lack of stub files

  File "PyInstaller/loader/pyimod02_importers.py", line 352, in exec_module
  File "skimage/__init__.py", line 74, in <module>
  File "lazy_loader/__init__.py", line 243, in attach_stub
ValueError: Cannot load imports from non-existent stub '/home/runner/work/PartSeg/PartSeg/dist/PartSeg/skimage/__init__.pyci'
[11749] Failed to execute script 'launcher_main' due to unhandled exception!

https://github.com/4DNucleome/PartSeg/actions/runs/4293660567/jobs/7481623224#step:10:183

Of course, I could try to workaround this by add more entries to package_data but I think that stub files in frozen applications are pointless.

Python allows to check if the application is frozen by getattr(sys, "frozen", False) so it may be easy to add a proper check and not raise an exception in such a use case.

stefanv commented 1 year ago

So the stub file cannot be bundled and accessed?

If so, then an exception sounds right. Are you able to make a PR? I don't have a setup to test this.

Czaki commented 1 year ago

So the stub file cannot be bundled and accessed?

It could be, but it needs to attach using datas that may need to be manually curated and As I read in the readme attach_stub is for IDE support which is not important in the frozen app.

If so, then an exception sounds right. Are you able to make a PR? I don't have a setup to test this.

I will do PR.

stefanv commented 1 year ago

Well, not exactly. The stubs are used to define the imports, so they are not optional and cannot be removed when attach_stub is used. We use __init__.pyi files to define the imports, vs __init__.py, so that Python doesn't immediately import modules; i.e., so we have a chance to load them lazily.

If you can ship the stub files inside your frozen archive, and if we can get access to them from the lazy loader, we can make it work.

Czaki commented 1 year ago

If you can ship the stub files inside your frozen archive, and if we can get access to them from the lazy loader, we can make it work.

It looks like collect_data_files("skimage") collect .pyi for skimage files so #39 and some update of readme may be enough, but need to investigate deeper.

stefanv commented 1 year ago

@Czaki Can you confirm whether or not your issue has been solved, and whether there's anything we need to do here still?