scrapinghub / shub

Scrapinghub Command Line Client
https://shub.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
125 stars 79 forks source link

AttributeError: 'PathDistribution' object has no attribute 'name' #441

Closed elacuesta closed 9 months ago

elacuesta commented 9 months ago

After #434 the following error appears when doing shub image deploy for some Python versions:

$ shub image deploy
Traceback (most recent call last):
  File "/.../venv/bin/shub", line 8, in <module>
    sys.exit(cli())
  File "/.../venv/lib/python3.8/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/.../venv/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../venv/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../venv/lib/python3.8/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/.../venv/lib/python3.8/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/.../venv/lib/python3.8/site-packages/shub/image/deploy.py", line 62, in cli
    deploy_cmd(target, version, username, password, email,
  File "/.../venv/lib/python3.8/site-packages/shub/image/deploy.py", line 79, in deploy_cmd
    params = _prepare_deploy_params(
  File "/.../venv/lib/python3.8/site-packages/shub/image/deploy.py", line 202, in _prepare_deploy_params
    metadata = list_mod.list_cmd(image_name, project, endpoint, apikey)
  File "/.../venv/lib/python3.8/site-packages/shub/image/list.py", line 67, in list_cmd
    exit_code, logs = _run_cmd_in_docker_container(
  File "/.../venv/lib/python3.8/site-packages/shub/image/list.py", line 107, in _run_cmd_in_docker_container
    client = utils.get_docker_client()
  File "/.../venv/lib/python3.8/site-packages/shub/image/utils.py", line 90, in get_docker_client
    if dep.name == 'docker-py':
AttributeError: 'PathDistribution' object has no attribute 'name'

Apparently the name attribute is new in Python 3.10, or maybe in a 3.9 version newer than 3.9.6 since that's the one I tested with:

Python 3.9.6 (default, Sep  6 2021, 10:09:19) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> d = list(metadata.distributions())
>>> type(d[0])
<class 'importlib.metadata.PathDistribution'>
>>> "name" in dir(d[0])
False
>>> 

Python 3.10.0 (default, Oct  8 2021, 09:55:22) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> d = list(metadata.distributions())
>>> type(d[0])                   
<class 'importlib.metadata.PathDistribution'>
>>> "name" in dir(d[0])
True
>>> 
wRAR commented 9 months ago

It's not present in my 3.9.13 either. Unfortunately the list of available fields doesn't seem to be documented, the Python API docs defer to various metadata docs which don't talk about Python versions.

One option for the fix is depending on importlib-metadata on Python < 3.10.

Gallaecio commented 9 months ago
tox -e py39 -- tests -k test_get_docker_client

Works for me with 3.9.6. And I am sure deps.name is called, coverage data confirms so and raising a ValueError right before makes that the outcome of the test.

But I can reproduce your steps in the REPL, which has me really confused.

wRAR commented 9 months ago

Is it possible that tox installs importlib-metadata?

Gallaecio commented 9 months ago

If I try to import importlib_metadata right before the dep.name line:

FAILED tests/image/test_utils.py::ReleaseUtilsTest::test_get_docker_client - ModuleNotFoundError: No module named 'importlib_metadata'
Gallaecio commented 9 months ago

I can reproduce the error in a venv calling get_docker_client, though, so tox is definitely doing something different.

Gallaecio commented 9 months ago

One option for the fix is depending on importlib-metadata on Python < 3.10.

I think this is the way to go. But I am concerned about not having a test for this and accidentally removing it in the future.

I wonder if we should have a special test out of tox that runs on CI.