Adding Googles (subpackage) protobuf fails

[MAKOMO]

My app imports a compiled protobuf specification

from proto import Spec_pb2

which itself does the following imports

from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database

Current py2app fails to include the Google protobuf subpackage even if I instruct to bundle it (as well as the dependent google._upb module) as whole package.

    'packages': [ 'google.protobuf', 'google._upb'],

On building I get the following error:

*** filtering dependencies ***
6605 total
49 filtered
194 orphaned
6556 remaining
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/dist.py", line 1244, in run_command
    super().run_command(command)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/py2app/build_app.py", line 984, in run
    self._run()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/py2app/build_app.py", line 1214, in _run
    self.run_normal()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/py2app/build_app.py", line 1326, in run_normal
    pkgdirs = self.collect_packagedirs()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/py2app/build_app.py", line 1290, in collect_packagedirs
    [
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/py2app/build_app.py", line 1291, in <listcomp>
    os.path.join(os.path.realpath(self.get_bootstrap(pkg)), "")
                                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/py2app/build_app.py", line 2202, in get_bootstrap
    bootstrap = imp_find_module(bootstrap)[1]
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/modulegraph/util.py", line 33, in imp_find_module
    result = imp.find_module(name, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/imp.py", line 297, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'google'

The fix is to add missing init.py files to the Google packages

# cd site-packages
# touch Google/__init__.py
# touch Google/protobuf/__init__.py
# touch Google/_upb/__init__.py

Does this relate to py2app doesn't work with subpackages

[ronaldoussoren]

I'll have to do some more debugging, but I did get a working bundle when using a script with

from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database

as part of the import statements. That's without using protobuf in the packages option.

Did you use an editable install for protobuf ("pip install -e protobuf")? That's unlikely, but I just noticed that py2app's (or rather modulegraph's) support for editable installs is broken due to changes in pip.

[MAKOMO]

I can confirm that without using the packages option the libs are correctly added to python311.zip. However, I couldn't get this signed as _upd/ contains the shared object file _message.abi3.so in the zip. That's why I went for the packages option.

I installed regular via

# pip protobuf