Avocado crashed during vt-bootstrap in python3.8

sathnaga commented 4 years ago

Latest versions of avocado, avocado-vt

avocado and avocado-vt installed using below steps as avocado-vt is not yet tagged for 82.0 used a HEAD-1 for avocado to force install 81.0

pip3 install -U git+https://github.com/avocado-framework/avocado.git@ff35265f889bbfca9257713ff25de0e08d776593
pip3 install -U git+https://github.com/avocado-framework/avocado-vt.git

avocado vt-bootstrap

Avocado crashed:
Traceback (most recent call last):
  File "/usr/local/bin/avocado", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/avocado/core/main.py", line 76, in main
    return app.run()
  File "/usr/local/lib/python3.8/dist-packages/avocado/core/app.py", line 112, in run
    return method(self.parser.config)
  File "/usr/local/lib/python3.8/dist-packages/avocado_vt/plugins/vt_bootstrap.py", line 100, in run
    bootstrap.bootstrap(options=config, interactive=True)
  File "/usr/local/lib/python3.8/dist-packages/virttest/bootstrap.py", line 890, in bootstrap
    dir_util.copy_tree(tp_base_dir, tp_local_dir)
  File "/usr/lib/python3.8/distutils/dir_util.py", line 126, in copy_tree
    raise DistutilsFileError(
distutils.errors.DistutilsFileError: cannot copy tree '/usr/local/lib/python3.8/dist-packages/test-providers.d': not a directory

the same versions of avocado, avocado-vt work fine in python3.9 environment, is this anything to do with specific python version?

sathnaga commented 4 years ago

looks like the issue is due to avocado-vt plugin installation is not populating /usr/share/avocado-plugins-vt in below environment, ubuntu 20.04,

# python3
Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

sathnaga commented 4 years ago

same works fine in other 3.7, 3.9.0rc1 envs Fedora 33


# python3
Python 3.9.0rc1 (default, Aug 12 2020, 00:00:00) 
[GCC 10.2.1 20200804 (Red Hat 10.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> ```

clebergnu commented 4 years ago

@sathnaga thanks for reporting this, I'll take a look and try to reproduce it.

willianrampazzo commented 4 years ago

I could not reproduce it on:

Python 3.8.5 (default, Aug 12 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux

pevogam commented 4 years ago

I get the same problem in Travis CI of the Avocado I2N plugin:

$ avocado vt-bootstrap --vt-skip-verify-download-assets --yes-to-all

Failed to load plugin from module "avocado_vt.plugins.vt": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :

  File "/home/travis/build/intra2net/avocado-i2n/avocado-libs/avocado/core/extension_manager.py", line 62, in __init__

    plugin = ep.load()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load

    return self.resolve()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve

    module = __import__(self.module_name, fromlist=['__name__'], level=0)

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>

    raise ImportError("Bootstrap missing. "

Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :

  File "/home/travis/build/intra2net/avocado-i2n/avocado-libs/avocado/core/extension_manager.py", line 62, in __init__

    plugin = ep.load()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load

    return self.resolve()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve

    module = __import__(self.module_name, fromlist=['__name__'], level=0)

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/avocado_vt/plugins/vt_list.py", line 26, in <module>

    from .vt import add_basic_vt_options, add_qemu_bin_vt_option

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>

    raise ImportError("Bootstrap missing. "

Running bootstrap for qemu

1 - Checking the mandatory programs and headers

/usr/bin/xz OK

/usr/sbin/tcpdump OK

/bin/nc OK

/bin/ip OK

/usr/sbin/arping OK

/usr/bin/gcc OK

/usr/include/graphviz/types.h OK

/usr/include/python3.5/Python.h OK

/usr/include/linux/socket.h OK

/usr/include/linux/unistd.h OK

2 - Checking the recommended programs

Recommended command missing. You may want to install it if not building it from source. Aliases searched: ('qemu-kvm', 'kvm')

Recommended command qemu-img missing. You may want to install it if not building from source.

Recommended command qemu-io missing. You may want to install it if not building from source.

Recommended command virsh missing. You may want to install it if not building from source.

3 - Updating test providers repo configuration from local copy

Avocado crashed unexpectedly: cannot copy tree '/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/test-providers.d': not a directory

Full log at: https://travis-ci.org/github/intra2net/avocado-i2n/jobs/728417905

willianrampazzo commented 4 years ago

I get the same problem in Travis CI of the Avocado I2N plugin:

$ avocado vt-bootstrap --vt-skip-verify-download-assets --yes-to-all

Failed to load plugin from module "avocado_vt.plugins.vt": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :

  File "/home/travis/build/intra2net/avocado-i2n/avocado-libs/avocado/core/extension_manager.py", line 62, in __init__

    plugin = ep.load()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load

    return self.resolve()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve

    module = __import__(self.module_name, fromlist=['__name__'], level=0)

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>

    raise ImportError("Bootstrap missing. "

Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :

  File "/home/travis/build/intra2net/avocado-i2n/avocado-libs/avocado/core/extension_manager.py", line 62, in __init__

    plugin = ep.load()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load

    return self.resolve()

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve

    module = __import__(self.module_name, fromlist=['__name__'], level=0)

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/avocado_vt/plugins/vt_list.py", line 26, in <module>

    from .vt import add_basic_vt_options, add_qemu_bin_vt_option

  File "/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>

    raise ImportError("Bootstrap missing. "

Running bootstrap for qemu

1 - Checking the mandatory programs and headers

/usr/bin/xz OK

/usr/sbin/tcpdump OK

/bin/nc OK

/bin/ip OK

/usr/sbin/arping OK

/usr/bin/gcc OK

/usr/include/graphviz/types.h OK

/usr/include/python3.5/Python.h OK

/usr/include/linux/socket.h OK

/usr/include/linux/unistd.h OK

2 - Checking the recommended programs

Recommended command missing. You may want to install it if not building it from source. Aliases searched: ('qemu-kvm', 'kvm')

Recommended command qemu-img missing. You may want to install it if not building from source.

Recommended command qemu-io missing. You may want to install it if not building from source.

Recommended command virsh missing. You may want to install it if not building from source.

3 - Updating test providers repo configuration from local copy

Avocado crashed unexpectedly: cannot copy tree '/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/test-providers.d': not a directory

Full log at: https://travis-ci.org/github/intra2net/avocado-i2n/jobs/728417905

@pevogam I think your error messages are not related to the problem reported by @sathnaga.

pevogam commented 4 years ago

@willianrampazzo The error I get is recent and has the same error message so it is reasonable to believe it is related but I could be wrong.

sathnaga commented 4 years ago

Avocado crashed unexpectedly: cannot copy tree '/home/travis/virtualenv/python3.8.0/lib/python3.8/site-packages/test-providers.d': not a directory

it is the same issue @pevogam @willianrampazzo

sathnaga commented 4 years ago

able to recreate in f33 version aswell, cleanup the environment

[root@9 ~]#rm -rf /usr/share/avocado-plugins-vt

[root@9 ~]# pip search avocado
avocado-framework (82.0)                               - Avocado Test Framework
avocado-epigenome (0.4.0)                              - Avocado is a package for learning a latent representation of the human epigenome.
avocado-framework-plugin-i2n (81.0)                    - Avocado Intra2net Plugins
avocado-framework-plugin-vt (82.0)                     - Avocado Plugin for Virtualization Testing
avocado (2.4.3)                                        - Metadata APIs for Django
avocado-framework-plugin-golang (82.0)                 - Avocado Plugin for Execution of Golang tests
avocado-framework-plugin-robot (82.0)                  - Avocado Plugin for Execution of Robot Framework tests
avocado-framework-plugin-resultsdb (82.0)              - Avocado Plugin to propagate Job results to Resultsdb
avocado-framework-plugin-result-html (82.0)            - Avocado HTML Report for Jobs
avocado-framework-plugin-runner-remote (80.0)          - Avocado Runner for Remote Execution
avocado-framework-plugin-glib (82.0)                   - Avocado Plugin for Execution of GLib Test Framework tests
avocado-framework-plugin-runner-vm (80.0)              - Avocado Runner for libvirt VM Execution
avocado-framework-plugin-runner-docker (80.0)          - Avocado Runner for Execution on Docker Containers
avocado-framework-plugin-result-upload (82.0)          - Avocado Plugin to propagate Job results to remote host
avocado-framework-plugin-loader-yaml (82.0)            - Avocado Plugin that loads tests from YAML files
Django-Avocado (0.2.0)                                 - Delicious delayed and cached database-logging for django.
avocado-framework-plugin-varianter-yaml-to-mux (82.0)  - Avocado Varianter plugin to parse YAML file into variants
avocado-messenger-server (0.0.2)                       - Server for avocado_messenger
serrano (2.4.4)                                        - Hypermedia implementation for Avocado
AvocadoPricePredictionModel (0.3.7)                    - A model to predict avocado prices!
avocado-framework-plugin-varianter-pict (82.0)         - Varianter with combinatorial capabilities by PICT
avocado-framework-plugin-varianter-cit (82.0)          - Varianter with combinatorial capabilities
dinolemma (0.0.12)                                     - Simulate evolution of dinosaurs and avocado trees

Nothing installed on the system, let's start with clean slate.

python version:
[root@9 ~]# python3
Python 3.9.0rc1 (default, Aug 12 2020, 00:00:00) 
[GCC 10.2.1 20200804 (Red Hat 10.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.

Steps to recreate:


[root@9 ~]# pip3 install avocado-framework
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting avocado-framework
  Using cached avocado_framework-82.0-py3-none-any.whl (385 kB)
Requirement already satisfied: setuptools in /usr/lib/python3.9/site-packages (from avocado-framework) (49.1.3)
Installing collected packages: avocado-framework
Successfully installed avocado-framework-82.0

[root@9 ~]# pip3 install avocado-framework-plugin-vt
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting avocado-framework-plugin-vt
  Using cached avocado_framework_plugin_vt-82.0-py3-none-any.whl (8.3 MB)
Requirement already satisfied: avocado-framework>=68.0 in /usr/local/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (82.0)
Requirement already satisfied: netifaces in /usr/local/lib64/python3.9/site-packages (from avocado-framework-plugin-vt) (0.10.9)
Requirement already satisfied: aexpect in /usr/local/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (1.5.1)
Requirement already satisfied: six in /usr/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (1.15.0)
Requirement already satisfied: netaddr in /usr/local/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (0.8.0)
Requirement already satisfied: simplejson in /usr/local/lib64/python3.9/site-packages (from avocado-framework-plugin-vt) (3.17.2)
Requirement already satisfied: setuptools in /usr/lib/python3.9/site-packages (from avocado-framework>=68.0->avocado-framework-plugin-vt) (49.1.3)
Installing collected packages: avocado-framework-plugin-vt
Successfully installed avocado-framework-plugin-vt-82.0

[root@9 ~]# avocado vt-bootstrap
Failed to load plugin from module "avocado_vt.plugins.vt": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :
  File "/usr/local/lib/python3.9/site-packages/avocado/core/extension_manager.py", line 62, in __init__
    plugin = ep.load()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2461, in load
    return self.resolve()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2467, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>
    raise ImportError("Bootstrap missing. "

Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :
  File "/usr/local/lib/python3.9/site-packages/avocado/core/extension_manager.py", line 62, in __init__
    plugin = ep.load()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2461, in load
    return self.resolve()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2467, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt_list.py", line 26, in <module>
    from .vt import add_basic_vt_options, add_qemu_bin_vt_option
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>
    raise ImportError("Bootstrap missing. "

Running bootstrap for qemu

1 - Checking the mandatory programs and headers
/bin/xz OK
/sbin/tcpdump OK
/bin/nc OK
/sbin/ip OK
/bin/arping OK
/bin/gcc OK
/usr/include/sys/unistd.h OK
/usr/include/sys/types.h OK
/usr/include/sys/socket.h OK
/usr/include/python3.9/Python.h OK

2 - Checking the recommended programs
/bin/qemu-kvm OK
/usr/local/bin/qemu-img OK
/usr/local/bin/qemu-io OK
/bin/virsh OK

3 - Updating test providers repo configuration from local copy
Avocado crashed unexpectedly: cannot copy tree '/usr/local/lib/python3.9/site-packages/test-providers.d': not a directory
You can find details in /home/sath/tests/data/crashes/avocado-traceback-2020-09-21_05:19:46-b97fojet.log

[root@9 ~]# cat /home/sath/tests/data/crashes/avocado-traceback-2020-09-21_05:19:46-b97fojet.log
Avocado crashed:
Traceback (most recent call last):
  File "/usr/local/bin/avocado", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/avocado/core/main.py", line 76, in main
    return app.run()
  File "/usr/local/lib/python3.9/site-packages/avocado/core/app.py", line 112, in run
    return method(self.parser.config)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt_bootstrap.py", line 100, in run
    bootstrap.bootstrap(options=config, interactive=True)
  File "/usr/local/lib/python3.9/site-packages/virttest/bootstrap.py", line 890, in bootstrap
    dir_util.copy_tree(tp_base_dir, tp_local_dir)
  File "/usr/lib64/python3.9/distutils/dir_util.py", line 123, in copy_tree
    raise DistutilsFileError(
distutils.errors.DistutilsFileError: cannot copy tree '/usr/local/lib/python3.9/site-packages/test-providers.d': not a directory

sathnaga commented 4 years ago

@pevogam tried to fix it via, avocado-framework/avocado-vt#2769, feel free to try in your environment and check if it fixes, thanks!

@clebergnu @willianrampazzo fyi...

willianrampazzo commented 4 years ago

able to recreate in f33 version aswell, cleanup the environment

[root@9 ~]#rm -rf /usr/share/avocado-plugins-vt

[root@9 ~]# pip search avocado
avocado-framework (82.0)                               - Avocado Test Framework
avocado-epigenome (0.4.0)                              - Avocado is a package for learning a latent representation of the human epigenome.
avocado-framework-plugin-i2n (81.0)                    - Avocado Intra2net Plugins
avocado-framework-plugin-vt (82.0)                     - Avocado Plugin for Virtualization Testing
avocado (2.4.3)                                        - Metadata APIs for Django
avocado-framework-plugin-golang (82.0)                 - Avocado Plugin for Execution of Golang tests
avocado-framework-plugin-robot (82.0)                  - Avocado Plugin for Execution of Robot Framework tests
avocado-framework-plugin-resultsdb (82.0)              - Avocado Plugin to propagate Job results to Resultsdb
avocado-framework-plugin-result-html (82.0)            - Avocado HTML Report for Jobs
avocado-framework-plugin-runner-remote (80.0)          - Avocado Runner for Remote Execution
avocado-framework-plugin-glib (82.0)                   - Avocado Plugin for Execution of GLib Test Framework tests
avocado-framework-plugin-runner-vm (80.0)              - Avocado Runner for libvirt VM Execution
avocado-framework-plugin-runner-docker (80.0)          - Avocado Runner for Execution on Docker Containers
avocado-framework-plugin-result-upload (82.0)          - Avocado Plugin to propagate Job results to remote host
avocado-framework-plugin-loader-yaml (82.0)            - Avocado Plugin that loads tests from YAML files
Django-Avocado (0.2.0)                                 - Delicious delayed and cached database-logging for django.
avocado-framework-plugin-varianter-yaml-to-mux (82.0)  - Avocado Varianter plugin to parse YAML file into variants
avocado-messenger-server (0.0.2)                       - Server for avocado_messenger
serrano (2.4.4)                                        - Hypermedia implementation for Avocado
AvocadoPricePredictionModel (0.3.7)                    - A model to predict avocado prices!
avocado-framework-plugin-varianter-pict (82.0)         - Varianter with combinatorial capabilities by PICT
avocado-framework-plugin-varianter-cit (82.0)          - Varianter with combinatorial capabilities
dinolemma (0.0.12)                                     - Simulate evolution of dinosaurs and avocado trees

Nothing installed on the system, let's start with clean slate.

python version:
[root@9 ~]# python3
Python 3.9.0rc1 (default, Aug 12 2020, 00:00:00) 
[GCC 10.2.1 20200804 (Red Hat 10.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.

Steps to recreate:

[root@9 ~]# pip3 install avocado-framework
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting avocado-framework
  Using cached avocado_framework-82.0-py3-none-any.whl (385 kB)
Requirement already satisfied: setuptools in /usr/lib/python3.9/site-packages (from avocado-framework) (49.1.3)
Installing collected packages: avocado-framework
Successfully installed avocado-framework-82.0

[root@9 ~]# pip3 install avocado-framework-plugin-vt
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting avocado-framework-plugin-vt
  Using cached avocado_framework_plugin_vt-82.0-py3-none-any.whl (8.3 MB)
Requirement already satisfied: avocado-framework>=68.0 in /usr/local/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (82.0)
Requirement already satisfied: netifaces in /usr/local/lib64/python3.9/site-packages (from avocado-framework-plugin-vt) (0.10.9)
Requirement already satisfied: aexpect in /usr/local/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (1.5.1)
Requirement already satisfied: six in /usr/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (1.15.0)
Requirement already satisfied: netaddr in /usr/local/lib/python3.9/site-packages (from avocado-framework-plugin-vt) (0.8.0)
Requirement already satisfied: simplejson in /usr/local/lib64/python3.9/site-packages (from avocado-framework-plugin-vt) (3.17.2)
Requirement already satisfied: setuptools in /usr/lib/python3.9/site-packages (from avocado-framework>=68.0->avocado-framework-plugin-vt) (49.1.3)
Installing collected packages: avocado-framework-plugin-vt
Successfully installed avocado-framework-plugin-vt-82.0

[root@9 ~]# avocado vt-bootstrap
Failed to load plugin from module "avocado_vt.plugins.vt": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :
  File "/usr/local/lib/python3.9/site-packages/avocado/core/extension_manager.py", line 62, in __init__
    plugin = ep.load()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2461, in load
    return self.resolve()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2467, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>
    raise ImportError("Bootstrap missing. "

Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") :
  File "/usr/local/lib/python3.9/site-packages/avocado/core/extension_manager.py", line 62, in __init__
    plugin = ep.load()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2461, in load
    return self.resolve()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2467, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt_list.py", line 26, in <module>
    from .vt import add_basic_vt_options, add_qemu_bin_vt_option
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt.py", line 40, in <module>
    raise ImportError("Bootstrap missing. "

I was able to reproduce this error message.

Running bootstrap for qemu

1 - Checking the mandatory programs and headers
/bin/xz OK
/sbin/tcpdump OK
/bin/nc OK
/sbin/ip OK
/bin/arping OK
/bin/gcc OK
/usr/include/sys/unistd.h OK
/usr/include/sys/types.h OK
/usr/include/sys/socket.h OK
/usr/include/python3.9/Python.h OK

2 - Checking the recommended programs
/bin/qemu-kvm OK
/usr/local/bin/qemu-img OK
/usr/local/bin/qemu-io OK
/bin/virsh OK

3 - Updating test providers repo configuration from local copy
Avocado crashed unexpectedly: cannot copy tree '/usr/local/lib/python3.9/site-packages/test-providers.d': not a directory
You can find details in /home/sath/tests/data/crashes/avocado-traceback-2020-09-21_05:19:46-b97fojet.log

But I cannot reproduce this error message. That is why I mentioned I don't think they are related.

[root@9 ~]# cat /home/sath/tests/data/crashes/avocado-traceback-2020-09-21_05:19:46-b97fojet.log
Avocado crashed:
Traceback (most recent call last):
  File "/usr/local/bin/avocado", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/avocado/core/main.py", line 76, in main
    return app.run()
  File "/usr/local/lib/python3.9/site-packages/avocado/core/app.py", line 112, in run
    return method(self.parser.config)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt_bootstrap.py", line 100, in run
    bootstrap.bootstrap(options=config, interactive=True)
  File "/usr/local/lib/python3.9/site-packages/virttest/bootstrap.py", line 890, in bootstrap
    dir_util.copy_tree(tp_base_dir, tp_local_dir)
  File "/usr/lib64/python3.9/distutils/dir_util.py", line 123, in copy_tree
    raise DistutilsFileError(
distutils.errors.DistutilsFileError: cannot copy tree '/usr/local/lib/python3.9/site-packages/test-providers.d': not a directory

sathnaga commented 4 years ago

[root@9 ~]# avocado vt-bootstrap Failed to load plugin from module "avocado_vt.plugins.vt": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") : File "/usr/local/lib/python3.9/site-packages/avocado/core/extension_manager.py", line 62, in init plugin = ep.load() File "/usr/lib/python3.9/site-packages/pkg_resources/init.py", line 2461, in load return self.resolve() File "/usr/lib/python3.9/site-packages/pkg_resources/init.py", line 2467, in resolve module = import(self.module_name, fromlist=['name'], level=0) File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt.py", line 40, in raise ImportError("Bootstrap missing. "

Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError("Bootstrap missing. Execute 'avocado vt-bootstrap' or disable this plugin to get rid of this message") : File "/usr/local/lib/python3.9/site-packages/avocado/core/extension_manager.py", line 62, in init plugin = ep.load() File "/usr/lib/python3.9/site-packages/pkg_resources/init.py", line 2461, in load return self.resolve() File "/usr/lib/python3.9/site-packages/pkg_resources/init.py", line 2467, in resolve module = import(self.module_name, fromlist=['name'], level=0) File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt_list.py", line 26, in from .vt import add_basic_vt_options, add_qemu_bin_vt_option File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt.py", line 40, in raise ImportError("Bootstrap missing. "
I was able to reproduce this error message.

ah, okies, actually this is not an issue, you would see this very first time you do vt-bootstrap...

Running bootstrap for qemu

1 - Checking the mandatory programs and headers
/bin/xz OK
/sbin/tcpdump OK
/bin/nc OK
/sbin/ip OK
/bin/arping OK
/bin/gcc OK
/usr/include/sys/unistd.h OK
/usr/include/sys/types.h OK
/usr/include/sys/socket.h OK
/usr/include/python3.9/Python.h OK

2 - Checking the recommended programs
/bin/qemu-kvm OK
/usr/local/bin/qemu-img OK
/usr/local/bin/qemu-io OK
/bin/virsh OK

3 - Updating test providers repo configuration from local copy
Avocado crashed unexpectedly: cannot copy tree '/usr/local/lib/python3.9/site-packages/test-providers.d': not a directory
You can find details in /home/sath/tests/data/crashes/avocado-traceback-2020-09-21_05:19:46-b97fojet.log

But I cannot reproduce this error message. That is why I mentioned I don't think they are related.

this is actual issue we are discussing about and @pevogam error message also had this issue.

[root@9 ~]# cat /home/sath/tests/data/crashes/avocado-traceback-2020-09-21_05:19:46-b97fojet.log
Avocado crashed:
Traceback (most recent call last):
  File "/usr/local/bin/avocado", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/avocado/core/main.py", line 76, in main
    return app.run()
  File "/usr/local/lib/python3.9/site-packages/avocado/core/app.py", line 112, in run
    return method(self.parser.config)
  File "/usr/local/lib/python3.9/site-packages/avocado_vt/plugins/vt_bootstrap.py", line 100, in run
    bootstrap.bootstrap(options=config, interactive=True)
  File "/usr/local/lib/python3.9/site-packages/virttest/bootstrap.py", line 890, in bootstrap
    dir_util.copy_tree(tp_base_dir, tp_local_dir)
  File "/usr/lib64/python3.9/distutils/dir_util.py", line 123, in copy_tree
    raise DistutilsFileError(
distutils.errors.DistutilsFileError: cannot copy tree '/usr/local/lib/python3.9/site-packages/test-providers.d': not a directory

clebergnu commented 3 years ago

@sathnaga I believe this was already fixed by https://github.com/avocado-framework/avocado-vt/commit/ba632749d62355ffda3b48f64c70f2f8135433f6. Could you please verify? I tested with:

$ pip3 install -U git+https://github.com/avocado-framework/avocado.git@7d4aabee23152d7154c5d6278e3182511d67c466
$ pip3 install -U git+https://github.com/avocado-framework/avocado-vt.git@6ab11ab8260d7888d092e00b0254b8c3ae25d420
$ avocado vt-bootstrap

And had no issues. I'm closing as I'm pretty confident about the fix, but please reopen if necessary.

avocado-framework / avocado

Avocado crashed during vt-bootstrap in python3.8 #4235