search

ManageIQ / integration_tests

ManageIQ integration tests
GNU General Public License v2.0
69 stars 165 forks source link

[CFME][Automation][Python_Quickstart_script] – When trying to run the python ‘quickstart’ script using python3, the script fails #10335

Open ikanias opened 4 years ago

ikanias commented 4 years ago

Summary: [CFME][Automation][Python_Quickstart_script] – When trying to run the python ‘quickstart’ script using python3, the script fails

Bug severity: Critical

Description:

  1. Make sure you have Python 3.7 installed on your machine

  2. Configure the automation environment according to the following steps:

    • mkdir cfme • cd cfme • git clone https://github.com/ManageIQ/integration_tests/ • cd integration_tests/conf/ • echo >.yaml_key • python3 -m cfme.scripting.quickstart

Expected Result: The script will function properly and a new /cfme_venv folder will be created alongside the ‘integration_tests’ and the ‘cfme-qe-yamls’ folders. In addition the automation environment will be configured successfully.

Actual Result: When running the ‘python3 -m cfme.scripting.quickstart’ command there is an error message saying that the ‘Feature’ name could not be imported from 'setuptools' and the script fails.

The error received is:

QS $ yum install -y python2-virtualenv gcc postgresql-devel libxml2-devel libxslt-devel zeromq3-devel libcurl-devel redhat-rpm-config gcc-c++ openssl-devel libffi-devel python-devel tesseract freetype-devel    yum-utils
QS $ debuginfo-install -y python2 python3
INFO: Virtualenv .cfme_venv already exists, skipping creation
QS $ .cfme_venv/bin/pip install -r requirements/frozen.py3.txt --no-binary pycurl --no-binary azure-cosmosdb-table -q # this may take some time to finish ...
    ERROR: Command errored out with exit status 1:
     command: /root/cfme/integration_tests/.cfme_venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sxg3vblj/markupsafe/setup.py'"'"'; __file__='"'"'/tmp/pip-install-sxg3vblj/markupsafe/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-b2rlyb7h
         cwd: /tmp/pip-install-sxg3vblj/markupsafe/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-sxg3vblj/markupsafe/setup.py", line 6, in <module>
        from setuptools import setup, Extension, Feature
    ImportError: cannot import name 'Feature' from 'setuptools' (/root/cfme/integration_tests/.cfme_venv/lib/python3.7/site-packages/setuptools/__init__.py)
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available.
You should consider upgrading via the '/root/cfme/integration_tests/.cfme_venv/bin/python3 -m pip install --upgrade pip' command.
Running command failed!
CalledProcessError(1, ['.cfme_venv/bin/pip', 'install', '-r', 'requirements/frozen.py3.txt', '--no-binary', 'pycurl', '--no-binary', 'azure-cosmosdb-table', '-q'])
mshriver commented 4 years ago

From the output it looks like this is on a re-run of quickstart, as the venv already existed. What OS and version are you running on? Quickstart's first logged line is the identified OS and version, which I don't see in your output

ikanias commented 4 years ago

Hello Mike and thank you for your reply. I am using RHEL 7.5 and CFME 5.11.8.

Regards,

Itsik

On Wed, Oct 7, 2020, 17:23 Mike Shriver notifications@github.com wrote:

From the output it looks like this is on a re-run of quickstart, as the venv already existed. What OS and version are you running on? Quickstart's first logged line is the identified OS and version, which I don't see in your output

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ManageIQ/integration_tests/issues/10335#issuecomment-704971448, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQUQ6XH2UX6YIDXTAEDKPYDSJR2WZANCNFSM4R6GFXOA .

jawatts commented 4 years ago

Can you try removing cfme_venv and rerunning again. Please paste the entire output

mshriver commented 4 years ago

trying to recreate - btw RHEL 7.5 was EOL back in April, might be time for an upgrade.

ikanias commented 4 years ago

Hi,

There is no cfme_venv folder. That's the strange thing... [image: image.png]

On Wed, Oct 7, 2020 at 8:58 PM Mike Shriver notifications@github.com wrote:

trying to recreate - btw RHEL 7.5 was EOL back in April, might be time for an upgrade.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ManageIQ/integration_tests/issues/10335#issuecomment-705099772, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQUQ6XEJ3TV5YIOLVDMK5X3SJST2PANCNFSM4R6GFXOA .

andyvesel commented 4 years ago

@ikanias it looks like your development env hasn't been activated. Activate the development environment by . ../cfme_venv/bin/activate Check this out ^

ikanias commented 4 years ago

I cannot activate it because the script 'python3 -m cfme.scripting.quickstart' that should create the environment is not working (That's what this issue was opened for).

andyvesel commented 4 years ago

I tried to make steps from https://cfme-tests.readthedocs.io/en/master/getting_started.html, and run quickstart. Even though I haven't run further steps (selenium server, etc), the script itself worked without errors. 

~/c/integration_tests (master|…) $ python -m cfme.scripting.quickstart                                                                                        Thu 08 Oct 2020 11:21:50 AM CEST
OS_NAME: Fedora, OS_VERSION: 31
QS $ sudo dnf install -y gcc postgresql-devel libxml2-devel libxslt-devel zeromq-devel libcurl-devel redhat-rpm-config gcc-c++ openssl-devel libffi-devel python3 python3-pip python3-devel tesseract freetype-devel  git 'dnf-command(debuginfo-install)'
[sudo] password for aveselov: 
INFO: Virtualenv .cfme_venv already exists, skipping creation
QS $ .cfme_venv/bin/pip3 install -U pip wheel setuptools setuptools_scm docutils pbr
INFO: skipping requirement installation as frozen ones didn't change
      to enforce please invoke pip manually
QS $ .cfme_venv/bin/python3 -m cfme.scripting.disable_bytecode
QS $ .cfme_venv/bin/pip3 install -q -e .
QS $ .cfme_venv/bin/python3 -m cfme.scripting.link_config ../cfme-qe-yamls/complete/ conf
QS $ .cfme_venv/bin/python3 -c 'import curl'
INFO: please remember to activate the virtualenv via
      . .cfme_venv/bin/activate

Please make sure that you have made touch conf/yaml.key and cp conf/env.yaml.template conf/env.yaml before running the script

ikanias commented 4 years ago

Try to run it with Python3 not Python. The problem happens when running the script with 'python3' at the beginning of the script instead of 'python'. Please make sure you have Python 3.7 installed.

Regards,

Itsik

On Thu, Oct 8, 2020, 14:45 Andrei Veselov notifications@github.com wrote:

I tried to make steps from https://cfme-tests.readthedocs.io/en/master/getting_started.html, and quickstart. Even though I haven't run further steps (selenium server, etc), the script itself worked without errors.

~/c/integration_tests (master|…) $ python -m cfme.scripting.quickstart Thu 08 Oct 2020 11:21:50 AM CEST

OS_NAME: Fedora, OS_VERSION: 31

QS $ sudo dnf install -y gcc postgresql-devel libxml2-devel libxslt-devel zeromq-devel libcurl-devel redhat-rpm-config gcc-c++ openssl-devel libffi-devel python3 python3-pip python3-devel tesseract freetype-devel git 'dnf-command(debuginfo-install)'

[sudo] password for aveselov:

INFO: Virtualenv .cfme_venv already exists, skipping creation

QS $ .cfme_venv/bin/pip3 install -U pip wheel setuptools setuptools_scm docutils pbr

INFO: skipping requirement installation as frozen ones didn't change

  to enforce please invoke pip manually

QS $ .cfme_venv/bin/python3 -m cfme.scripting.disable_bytecode

QS $ .cfme_venv/bin/pip3 install -q -e .

QS $ .cfme_venv/bin/python3 -m cfme.scripting.link_config ../cfme-qe-yamls/complete/ conf

QS $ .cfme_venv/bin/python3 -c 'import curl'

INFO: please remember to activate the virtualenv via

  . .cfme_venv/bin/activate

Please make sure that you have made touch conf/yaml.key and cp conf/env.yaml.template conf/env.yaml before running the script

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ManageIQ/integration_tests/issues/10335#issuecomment-705514806, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQUQ6XHYPUFXRRBVOVYIVNDSJWQ6DANCNFSM4R6GFXOA .

ikanias commented 4 years ago

Re-installed my BM with RHEL 7.9 and CFME 5.11.9.0 and Python 3.7.9. Then needed to install docker too. After all of that ran /usr/src/Python-3.7.9/python -m cfme.scripting.quickstart and it worked. Ran '. ../cfme_venv/bin/activate' then 'docker pull cfmeqe/sel_ff_chrome' and then ran 'miq selenium-container'. Then I ran 'miq-runtest --collect-only' and got errors probably because no cloning of 'https://gitlab.cee.redhat.com/cfme-qe/cfme-qe-yamls/' was done. Currently I have a certificate issue - working on it.