search

ceph / ceph-salt

Deploy Ceph clusters using cephadm
MIT License
31 stars 20 forks source link

ceph-salt "tox -e lint" broken for python 3.8 #148

Closed toabctl closed 4 years ago

toabctl commented 4 years ago

When I try to run tox -elint on openSUSE Tumbleweed with python 3.8, I get:

$ tox -relint
lint create: /home/tom/devel/ceph/ceph-salt/.tox/lint
lint installdeps: -r/home/tom/devel/ceph/ceph-salt/requirements.txt
ERROR: invocation failed (exit code 1), logfile: /home/tom/devel/ceph/ceph-salt/.tox/lint/log/lint-1.log
========================================================================= log start ==========================================================================
Processing /home/tom/devel/ceph/ceph-salt
Collecting mock==3.0.5
  Using cached https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl
Collecting pycodestyle==2.5.0
  Using cached https://files.pythonhosted.org/packages/0e/0c/04a353e104d2f324f8ee5f4b32012618c1c86dd79e52a433b64fceed511b/pycodestyle-2.5.0-py2.py3-none-any.whl
Collecting pyfakefs==3.7
  Using cached https://files.pythonhosted.org/packages/31/5a/bbf8a22ee586482a339712b9b43f1df1a4c17d76a06384a27dac7b477065/pyfakefs-3.7-py2.py3-none-any.whl
Collecting pylint==2.4.4
  Using cached https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl
Collecting pytest==5.3.1
  Using cached https://files.pythonhosted.org/packages/da/ed/d22d7f06eb1107271694ed2171b9d52e8eea38d9757124e75ba13324ac77/pytest-5.3.1-py3-none-any.whl
Collecting pytest-cov==2.8.1
  Using cached https://files.pythonhosted.org/packages/b9/54/3673ee8be482f81527678ac894276223b9814bb7262e4f730469bb7bf70e/pytest_cov-2.8.1-py2.py3-none-any.whl
Collecting pytest-runner==5.2
  Using cached https://files.pythonhosted.org/packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
Collecting Click>=6.7
  Using cached https://files.pythonhosted.org/packages/dd/c0/4d8f43a9b16e289f36478422031b8a63b54b6ac3b1ba605d602f10dd54d6/click-7.1.1-py2.py3-none-any.whl
Collecting configshell-fb>=1.1
  Using cached https://files.pythonhosted.org/packages/c1/b6/86599b430f70b4c8a77e47188fa0bcecca62ecd2900e1011ff37107dc18e/configshell-fb-1.1.27.tar.gz
Collecting pycryptodomex>=3.4.6
  Using cached https://files.pythonhosted.org/packages/45/80/5ec3419a504503cf00c984027f75ce683c553b98a458d7f5776f50c3d775/pycryptodomex-3.9.7-cp38-cp38-manylinux1_x86_64.whl
Processing /home/tom/.cache/pip/wheels/a7/c1/ea/cf5bd31012e735dc1dfea3131a2d5eae7978b251083d6247bd/PyYAML-5.3.1-cp38-cp38-linux_x86_64.whl
Collecting salt<3000
  Using cached https://files.pythonhosted.org/packages/25/ee/cd9ed4a912506f5f6f0eb00891e661840bd9df0fae781f78fa9f04515447/salt-2019.2.3.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /home/tom/devel/ceph/ceph-salt/.tox/lint/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-xps4yctn/salt/setup.py'"'"'; __file__='"'"'/tmp/pip-install-xps4yctn/salt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-xps4yctn/salt/pip-egg-info
         cwd: /tmp/pip-install-xps4yctn/salt/
    Complete output (14 lines):
    Traceback (most recent call last):
      File "/tmp/pip-install-xps4yctn/salt/salt/version.py", line 15, in <module>
        from platform import linux_distribution as _deprecated_linux_distribution
    ImportError: cannot import name 'linux_distribution' from 'platform' (/usr/lib64/python3.8/platform.py)

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-xps4yctn/salt/setup.py", line 137, in <module>
        exec(compile(open(SALT_VERSION).read(), SALT_VERSION, 'exec'))
      File "/tmp/pip-install-xps4yctn/salt/salt/version.py", line 22, in <module>
        from distro import linux_distribution
    ModuleNotFoundError: No module named 'distro'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 19.3.1; however, version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

========================================================================== log end ===========================================================================
ERROR: could not install deps [-r/home/tom/devel/ceph/ceph-salt/requirements.txt]; v = InvocationError('/home/tom/devel/ceph/ceph-salt/.tox/lint/bin/pip install -r/home/tom/devel/ceph/ceph-salt/requirements.txt', 1)
__________________________________________________________________________ summary ___________________________________________________________________________
ERROR:   lint: could not install deps [-r/home/tom/devel/ceph/ceph-salt/requirements.txt]; v = InvocationError('/home/tom/devel/ceph/ceph-salt/.tox/lint/bin/pip install -r/home/tom/devel/ceph/ceph-salt/requirements.txt', 1)

So salt itself needs to be fixed. See also https://github.com/saltstack/salt/issues/55310 and https://github.com/saltstack/salt/issues/55835

smithfarm commented 4 years ago

For testing the "next" Python version it would be nice to have a Vagrant box. For me, the ideal would be an openSUSE 15.2 Vagrant box that has Python 3.8 instead of 3.6.10.

@toabctl Do you know if such a Vagrant box is built by the OBS?

smithfarm commented 4 years ago

I changed the title to more accurately describe what is broken.

(I tested ceph-salt itself (installed from source) on Tumbleweed with Python 3.8 and it worked flawlessly.)

smithfarm commented 4 years ago

I can reproduce this bug. Here's the detailed reproducer:

  1. Install sesdev from https://github.com/SUSE/sesdev/pull/221
  2. sesdev create octopus --ceph-salt-repo https://github.com/ceph/ceph-salt --ceph-salt-branch master --os tumbleweed --single-node --qa-test octopus
  3. sesdev ssh octopus
  4. zypper in -n python3-tox
  5. cd ceph-salt
  6. tox -e lint

It also happens with salt-3000. The salt-3000 reproducer is the same as above, except for step 2:

  1. sesdev create octopus --ceph-salt-repo https://github.com/ricardoasmarques/ceph-salt --ceph-salt-branch salt-3000 --repo http://download.opensuse.org/repositories/systemsmanagement:/saltstack:/products:/next:/testing/openSUSE_Tumbleweed/ --no-repo-priority --os tumbleweed --single-node --qa-test octopus
smithfarm commented 4 years ago

And here is a minimal reproducer:

  1. sesdev create octopus --ceph-salt-repo https://github.com/ceph/ceph-salt --ceph-salt-branch master --stop-before-ceph-salt-config --os tumbleweed --single-node octopus
  2. sesdev ssh octopus
  3. zypper in -n python3-tox
  4. cd ceph-salt
  5. tox -e lint
toabctl commented 4 years ago

@toabctl Do you know if such a Vagrant box is built by the OBS?

No idea. I guess the Tumbleweed Vagrantbox has python3.8 now.

smithfarm commented 4 years ago

As proven by #338 this statement ("ceph-salt "tox -e lint" broken for python 3.8") no longer holds true.