regression with python3.7: AttributeError: 'zipimport.zipimporter' object has no attribute 'create_module'

[rggjan]

After updating to wrapt 1.14.0, I started seeing issues with pip installs on python3.7:

Installing build dependencies: started
Installing build dependencies: finished with status 'error'
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [58 lines of output]
    Traceback (most recent call last):
      File "/opt/miniconda/envs/foo/lib/python3.7/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/opt/miniconda/envs/foo/lib/python3.7/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/__main__.py", line 29, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/cli/main.py", line 9, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/cli/autocompletion.py", line 10, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/cli/main_parser.py", line 8, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/cli/cmdoptions.py", line 23, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/cli/parser.py", line 12, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/configuration.py", line 20, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_internal/exceptions.py", line 13, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_vendor/requests/__init__.py", line 135, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
      File "/tmp/pip-standalone-pip-8p809dqm/__env_pip__.zip/pip/_vendor/requests/utils.py", line 27, in <module>
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 670, in _load_unlocked
      File "<frozen importlib._bootstrap>", line 583, in module_from_spec
      File "/opt/miniconda/envs/shrek/lib/python3.7/site-packages/wrapt/importer.py", line 166, in create_module
        return self.loader.create_module(spec)
    AttributeError: 'zipimport.zipimporter' object has no attribute 'create_module'
    [end of output]

I assume it could be due to: https://github.com/GrahamDumpleton/wrapt/commit/53323ba4b92f88dab5a8ec3262a1c1f1e979a3e2

which introduced this code:

    def create_module(self, spec):
         return self.loader.create_module(spec)

However, the zipimporter module's create_module function was only added in python 3.10: https://docs.python.org/3/library/zipimport.html#zipimport.zipimporter.create_module

Reverting wrapt to 1.13.3 resolved the pip issues again.

[GrahamDumpleton]

As far as I am aware this is in part a problem with zimport version you are using not having been updated to new API. Can you use Python 3.10 instead, which finally updates zipimport? Still, I will see if I can find a way to deal somehow with other packages which haven't been updated to the new API. See also issue:

• https://github.com/GrahamDumpleton/wrapt/issues/209

[GrahamDumpleton]

FWIW, was only when you create this new issue that I realised that zipimport being talked about was the one in Python itself. I thought other reporter was talking about some third party package and that there was some newer version of it available that they started using. Didn't click that they updated Python version.

[rggjan]

Unfortunately, I'm stuck with the old python version due to other dependencies.

[GrahamDumpleton]

Don't suppose you have a small program that replicates the issue?

[rggjan]

Unfortunately I don't... also it happens when running a pip install

[GrahamDumpleton]

There is something dodgy going on if running pip install on the command line triggers it. That would imply something is force loading wrapt import hooks into arbitrary Python packages when run. Are you also using autowrapt by chance.

• https://pypi.org/project/autowrapt/

[rggjan]

Yes, this was run on commandline. Specifically, there was a conda base install, followed by a pip install ... which worked fine, followed by a second pip install ... which crashed.

No, I think autowrapt isn't used anywhere...

[rggjan]

Ah, I think I know what the issue could be. It seems like wrapt.when_imported is used on pip._vendor.requests to change some behaviour...

[GrahamDumpleton]

I still don't know how to replicate this, but can you try testing using:

pip install --no-cache-dir https://github.com/GrahamDumpleton/wrapt/archive/refs/heads/develop.zip

[rggjan]

Thanks @GrahamDumpleton

It's a bit hard to say as the issues comes from a conda installed package on my end, but I did some tests and it seems to work fine. Thanks!

[andrewleech]

I've run into this same issue in a package based on wrapt being used in an autowrap line way which targets pip / requests to make them use your system installed SSL certs rather than the ones bundled in requests; pip-system-certs

I don't have the wrapt version pinned on my dependencies (my bad) so you can get a reproduction of this issue by simply pip install pip-system-certs then trying to pip install <anything>

[andrewleech]

I see you've implemented a fix for this, I'll try to test it tomorrow thanks!

[GrahamDumpleton]

@andrewleech Did it work? I am still hoping someone can confirm the fix works before releasing the change.

[andrewleech]

Thanks for the reminder! I've just done a fresh test

python3 -m venv .
./bin/pip3 install pip-system-certs
./bin/pip3 uninstall pip-system-certs

Failed with

  File "/root/test2/lib/python3.8/site-packages/wrapt/importer.py", line 166, in create_module                                                                      return self.loader.create_module(spec)                                      AttributeError: 'zipimporter' object has no attribute 'create_module'

(Only on older python versions)

rm -rf *
python3 -m venv .
./bin/pip3 install git+https://github.com/GrahamDumpleton/wrapt.git
./bin/pip3 install pip-system-certs
./bin/pip3 uninstall pip-system-certs

Works perfectly

[GrahamDumpleton]

You would have to have done:

pip install -U https://github.com/GrahamDumpleton/wrapt/archive/refs/heads/develop.zip

immediately after creating the virtual environment and before installing pip-system-certs.

Did you do that?

[GrahamDumpleton]

Oh, wait I misunderstood the ordering of your statements. :-(

[andrewleech]

Sorry yes that was a rather terse description on my part - I first reproduced the issue by letting my package auto install the current release of wrapt.

I then reset the env, installed current master wrapt then installed my package, which then continued to work currently.

It definitely looks like your updates on master fixes the reproduction case I have of this issue.

[GrahamDumpleton]

Fix released in 1.14.1.