search

minishift / minishift-b2d-iso

ISO based on Boot2Docker for Minishift
Apache License 2.0
9 stars 13 forks source link

Issue #5 Add minishift smoke tests #18

Closed coolbrg closed 7 years ago

coolbrg commented 7 years ago

Fix #5

Should need changes from PR #13

coolbrg commented 7 years ago

@LalatenduMohanty Could you review PR?

LalatenduMohanty commented 7 years ago

@budhrg I am not able to install avocado framework on OS X. Looking in to the issue.

Lalatendus-MacBook-Pro:minishift-b2d-iso lmohanty$ pip install avocado-framework
Collecting avocado-framework
  Using cached avocado-framework-42.0.tar.gz
Collecting stevedore (from avocado-framework)
  Using cached stevedore-1.19.1-py2.py3-none-any.whl
Collecting six>=1.9.0 (from stevedore->avocado-framework)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting pbr>=1.8 (from stevedore->avocado-framework)
  Using cached pbr-1.10.0-py2.py3-none-any.whl
Installing collected packages: six, pbr, stevedore, avocado-framework
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/81/q3z841ds2yb7xblc91xww10m0000gn/T/pip-EXz0hj-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
hferentschik commented 7 years ago

I am not able to install avocado framework on OS X. Looking in to the issue.

Not me either. There are instructions missing. AFAIR, there were some customs steps needed for install Avocado on OS X. The QE had some info around that.

@LalatenduMohanty and even if you could install Avocado it would still not work for you, since other parts (docker-machine install) is Linux specific. So right now, tests will anyways only run on Linux. I am ok with that to move things along, but it should be fairly simple to get this to work on OS X as well. IMO, this could be done in a follow up issue. As long as it builds on CI I am fine for now

However, I had minishift smoke tests in mind, not docker-machine!

amitkrout commented 7 years ago

$ system_profiler SPSoftwareDataType

Software:

    System Software Overview:

      System Version: macOS 10.12.1 (16B2657)
      Kernel Version: Darwin 16.1.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: David’s MacBook Pro
      User Name: cdk qe team members (cdkqe)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Disabled
      Time since boot: 31 days 15 minutes

Avocado 42 installed without any error

$ pip install avocado-framework
Collecting avocado-framework
  Downloading avocado-framework-42.0.tar.gz (785kB)
    100% |████████████████████████████████| 788kB 940kB/s 
Requirement already satisfied (use --upgrade to upgrade): stevedore in /Library/Python/2.7/site-packages (from avocado-framework)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9.0 in /Library/Python/2.7/site-packages (from stevedore->avocado-framework)
Requirement already satisfied (use --upgrade to upgrade): pbr>=1.6 in /Library/Python/2.7/site-packages (from stevedore->avocado-framework)
Building wheels for collected packages: avocado-framework
  Running setup.py bdist_wheel for avocado-framework ... done
  Stored in directory: /Users/cdkqe/Library/Caches/pip/wheels/78/b3/94/3c8f8f441a8ebc527a327ff0d7dbfa93126718abc52e4c9fa0
Successfully built avocado-framework
Installing collected packages: avocado-framework
Successfully installed avocado-framework-42.0
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
coolbrg commented 7 years ago

@hferentschik @LalatenduMohanty @praveenkumar Updated the PR for minishift only tests.

Also, enabled running tests with showing logs on choice.

Once, we are fine with these tests, I will same for centos too. It's almost the exact copy/PR.

Log formats (Unfortunately only --show-job-log is available with avocado)

➜  minishift-b2d-iso git:(fix-5) ✗ make test      
avocado run  tests/test.py
JOB ID     : 01cd861849d2c032847621253f96be001e222af5
JOB LOG    : /home/budhram/avocado/job-results/job-2017-01-12T18.26-01cd861/job.log
TESTS      : 6
 (1/6) tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso: PASS (104.61 s)
 (2/6) tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm: PASS (0.15 s)
 (3/6) tests/test.py:MinishiftISOTest.test_ip_of_vm: PASS (0.06 s)
 (4/6) tests/test.py:MinishiftISOTest.test_docker_env_evaluable: PASS (0.06 s)
 (5/6) tests/test.py:MinishiftISOTest.test_stopping_vm: PASS (1.43 s)
 (6/6) tests/test.py:MinishiftISOTest.test_removing_vm: PASS (0.24 s)
RESULTS    : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
TESTS TIME : 106.55 s

# With SHOW_LOG switch 

➜  minishift-b2d-iso git:(fix-5) ✗ SHOW_LOG=--show-job-log make test
avocado run --show-job-log tests/test.py
Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
Command line: /usr/bin/avocado run --show-job-log tests/test.py

Avocado version: 44.0

Config files read (in order):
/etc/avocado/avocado.conf
/etc/avocado/conf.d/gdb.conf
/home/budhram/.config/avocado/avocado.conf

Avocado config:
Section.Key                             Value
datadir.paths.base_dir                  /usr/share/avocado
datadir.paths.test_dir                  /usr/share/avocado/tests
datadir.paths.data_dir                  /usr/share/avocado/data
datadir.paths.logs_dir                  ~/avocado/job-results
sysinfo.collect.enabled                 True
sysinfo.collect.installed_packages      False
sysinfo.collect.profiler                False
sysinfo.collect.locale                  C
sysinfo.collectibles.commands           /etc/avocado/sysinfo/commands
sysinfo.collectibles.files              /etc/avocado/sysinfo/files
sysinfo.collectibles.profilers          /etc/avocado/sysinfo/profilers
runner.output.colored                   True
runner.output.utf8                      
remoter.behavior.reject_unknown_hosts   False
remoter.behavior.disable_known_hosts    False
job.output.loglevel                     info
restclient.connection.hostname          localhost
restclient.connection.port              9405
restclient.connection.username          
restclient.connection.password          
plugins.disable                         []
plugins.skip_broken_plugin_notification []
plugins.loaders                         ['file', '@DEFAULT']
gdb.paths.gdb                           /usr/bin/gdb
gdb.paths.gdbserver                     /usr/bin/gdbserver

Avocado Data Directories:

Avocado replaces config dirs that can't be accessed
with sensible defaults. Please edit your local config
file to customize values

base     /home/budhram/avocado
tests    /home/budhram/avocado/tests
data     /home/budhram/avocado/data
logs     /home/budhram/avocado/job-results

Temporary dir: /var/tmp/avocado_CuNnYs

Variant 1:    /

Job ID: 34583f656c4b8c6e1988e62a2f5503a72c6f7864

Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
START 1-tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso
################################################################
Avocado version : 44.0
################################################################
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift start --vm-driver kvm --iso-url file:///home/budhram/redhat/minishift-b2d-iso/tests/../build/minishift-b2d.iso
PASS 1-tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso

Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
START 2-tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm
################################################################
Avocado version : 44.0
################################################################
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift ssh exit
PASS 2-tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm

Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
START 3-tests/test.py:MinishiftISOTest.test_ip_of_vm
################################################################
Avocado version : 44.0
################################################################
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift ip
PASS 3-tests/test.py:MinishiftISOTest.test_ip_of_vm

Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
START 4-tests/test.py:MinishiftISOTest.test_docker_env_evaluable
################################################################
Avocado version : 44.0
################################################################
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift docker-env
PASS 4-tests/test.py:MinishiftISOTest.test_docker_env_evaluable

Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
START 5-tests/test.py:MinishiftISOTest.test_stopping_vm
################################################################
Avocado version : 44.0
################################################################
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift stop
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift status
PASS 5-tests/test.py:MinishiftISOTest.test_stopping_vm

Commands configured by file: /etc/avocado/sysinfo/commands
Files configured by file: /etc/avocado/sysinfo/files
Profilers configured by file: /etc/avocado/sysinfo/profilers
Profiler disabled
System log file not found (looked for ['/var/log/messages', '/var/log/syslog', '/var/log/system.log'])
START 6-tests/test.py:MinishiftISOTest.test_removing_vm
################################################################
Avocado version : 44.0
################################################################
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift delete
Executing command : /home/budhram/redhat/minishift-b2d-iso/tests/../build/bin/minishift status
PASS 6-tests/test.py:MinishiftISOTest.test_removing_vm

Test results available in /home/budhram/avocado/job-results/job-2017-01-12T18.28-34583f6
coolbrg commented 7 years ago

Created the follow up issue https://github.com/minishift/minishift-b2d-iso/issues/20 to add support for Mac and Windows OS

coolbrg commented 7 years ago

Updated to fetch latest minishift version while downloading it.

coolbrg commented 7 years ago

@budhrg You need to decide whether you want this to just work for Linux now and CentOS CI and create a follow up issue or whether you want it to work for at least Linux + OS X now.

@hferentschik I have already created the follow up issue #20

praveenkumar commented 7 years ago

Something seems wrong to me. Check below result

$ make clean
rm -rf /home/prkumar/github/minishift-b2d-iso/build
rm -f /home/prkumar/github/minishift-b2d-iso/iso/os-release
[prkumar@cdk-prkumarws minishift-b2d-iso]$ make test
Downloading latest minishift binary...
######################################################################## 100.0%
Done.
avocado run  tests/test.py
JOB ID     : 72d31cec8472f775b861e9e772b729104595dad3
JOB LOG    : /home/prkumar/avocado/job-results/job-2017-01-13T18.10-72d31ce/job.log
TESTS      : 6
 (1/6) tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso: SKIP
 (2/6) tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm: SKIP
 (3/6) tests/test.py:MinishiftISOTest.test_ip_of_vm: SKIP
 (4/6) tests/test.py:MinishiftISOTest.test_docker_env_evaluable: SKIP
 (5/6) tests/test.py:MinishiftISOTest.test_stopping_vm: SKIP
 (6/6) tests/test.py:MinishiftISOTest.test_delete_vm: SKIP
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 6 | WARN 0 | INTERRUPT 0
TESTS TIME : 0.01 s

$ cat /home/prkumar/avocado/job-results/job-2017-01-13T18.10-72d31ce/job.log
2017-01-13 18:10:07,424 extension        L0156 DEBUG| found extension EntryPoint.parse('journal = avocado.plugins.journal:JournalResult')
2017-01-13 18:10:07,424 extension        L0156 DEBUG| found extension EntryPoint.parse('tap = avocado.plugins.tap:TAPResult')
2017-01-13 18:10:07,424 extension        L0156 DEBUG| found extension EntryPoint.parse('human = avocado.plugins.human:Human')
2017-01-13 18:10:07,427 extension        L0156 DEBUG| found extension EntryPoint.parse('jobscripts = avocado.plugins.jobscripts:JobScripts')
2017-01-13 18:10:07,428 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:07,428 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:07,428 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:07,428 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:08,105 job              L0300 INFO | Command line: /usr/bin/avocado run tests/test.py
2017-01-13 18:10:08,105 job              L0301 INFO | 
2017-01-13 18:10:08,105 job              L0306 INFO | Avocado version: 43.0
2017-01-13 18:10:08,106 job              L0318 INFO | 
2017-01-13 18:10:08,106 job              L0323 INFO | Config files read (in order):
2017-01-13 18:10:08,106 job              L0325 INFO | /etc/avocado/avocado.conf
2017-01-13 18:10:08,106 job              L0325 INFO | /etc/avocado/conf.d/gdb.conf
2017-01-13 18:10:08,106 job              L0330 INFO | 
2017-01-13 18:10:08,106 job              L0332 INFO | Avocado config:
2017-01-13 18:10:08,107 job              L0341 INFO | Section.Key                             Value
2017-01-13 18:10:08,107 job              L0341 INFO | datadir.paths.base_dir                  /usr/share/avocado
2017-01-13 18:10:08,107 job              L0341 INFO | datadir.paths.test_dir                  /usr/share/avocado/tests
2017-01-13 18:10:08,107 job              L0341 INFO | datadir.paths.data_dir                  /usr/share/avocado/data
2017-01-13 18:10:08,107 job              L0341 INFO | datadir.paths.logs_dir                  ~/avocado/job-results
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collect.enabled                 True
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collect.installed_packages      False
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collect.profiler                False
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collect.locale                  C
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collectibles.commands           /etc/avocado/sysinfo/commands
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collectibles.files              /etc/avocado/sysinfo/files
2017-01-13 18:10:08,107 job              L0341 INFO | sysinfo.collectibles.profilers          /etc/avocado/sysinfo/profilers
2017-01-13 18:10:08,107 job              L0341 INFO | runner.output.colored                   True
2017-01-13 18:10:08,107 job              L0341 INFO | runner.output.utf8                      
2017-01-13 18:10:08,107 job              L0341 INFO | runner.behavior.keep_tmp_files          False
2017-01-13 18:10:08,107 job              L0341 INFO | remoter.behavior.reject_unknown_hosts   False
2017-01-13 18:10:08,108 job              L0341 INFO | remoter.behavior.disable_known_hosts    False
2017-01-13 18:10:08,108 job              L0341 INFO | job.output.loglevel                     debug
2017-01-13 18:10:08,108 job              L0341 INFO | restclient.connection.hostname          localhost
2017-01-13 18:10:08,108 job              L0341 INFO | restclient.connection.port              9405
2017-01-13 18:10:08,108 job              L0341 INFO | restclient.connection.username          
2017-01-13 18:10:08,108 job              L0341 INFO | restclient.connection.password          
2017-01-13 18:10:08,108 job              L0341 INFO | plugins.disable                         []
2017-01-13 18:10:08,108 job              L0341 INFO | plugins.skip_broken_plugin_notification []
2017-01-13 18:10:08,108 job              L0341 INFO | plugins.loaders                         ['file', '@DEFAULT']
2017-01-13 18:10:08,108 job              L0341 INFO | gdb.paths.gdb                           /usr/bin/gdb
2017-01-13 18:10:08,108 job              L0341 INFO | gdb.paths.gdbserver                     /usr/bin/gdbserver
2017-01-13 18:10:08,108 job              L0342 INFO | 
2017-01-13 18:10:08,108 job              L0347 INFO | Avocado Data Directories:
2017-01-13 18:10:08,108 job              L0348 INFO | 
2017-01-13 18:10:08,108 job              L0349 INFO | Avocado replaces config dirs that can't be accessed
2017-01-13 18:10:08,108 job              L0350 INFO | with sensible defaults. Please edit your local config
2017-01-13 18:10:08,108 job              L0351 INFO | file to customize values
2017-01-13 18:10:08,109 job              L0352 INFO | 
2017-01-13 18:10:08,111 job              L0353 INFO | base     /home/prkumar/avocado
2017-01-13 18:10:08,111 job              L0354 INFO | tests    /home/prkumar/avocado/tests
2017-01-13 18:10:08,112 job              L0355 INFO | data     /home/prkumar/avocado/data
2017-01-13 18:10:08,112 job              L0356 INFO | logs     /home/prkumar/avocado/job-results
2017-01-13 18:10:08,112 job              L0357 INFO | 
2017-01-13 18:10:08,114 job              L0371 INFO | Temporary dir: /var/tmp/avocado_wOUEbA
2017-01-13 18:10:08,114 job              L0372 INFO | 
2017-01-13 18:10:08,114 job              L0379 INFO | Variant 1:    /
2017-01-13 18:10:08,114 job              L0382 INFO | 
2017-01-13 18:10:08,114 job              L0291 INFO | Job ID: 72d31cec8472f775b861e9e772b729104595dad3
2017-01-13 18:10:08,114 job              L0294 INFO | 
2017-01-13 18:10:08,177 sysinfo          L0107 DEBUG| Not logging /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor (file does not exist)
2017-01-13 18:10:08,280 sysinfo          L0107 DEBUG| Not logging /proc/pci (file does not exist)
2017-01-13 18:10:08,307 sysinfo          L0105 DEBUG| Not logging /proc/slabinfo (lack of permissions)
2017-01-13 18:10:08,318 sysinfo          L0107 DEBUG| Not logging /sys/kernel/debug/sched_features (file does not exist)
2017-01-13 18:10:08,449 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:08,450 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:08,450 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:08,450 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:08,458 multiplexer      L0168 DEBUG| PARAMS (key=timeout, path=*, default=None) => None
2017-01-13 18:10:08,458 test             L0217 INFO | START 1-tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso
2017-01-13 18:10:08,720 test             L0017 INFO | ################################################################
2017-01-13 18:10:08,720 test             L0018 INFO | Avocado version : 43.0
2017-01-13 18:10:08,720 test             L0019 INFO | ################################################################
2017-01-13 18:10:08,720 stacktrace       L0038 ERROR| 
2017-01-13 18:10:08,720 stacktrace       L0041 ERROR| Reproduced traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:436
2017-01-13 18:10:08,721 stacktrace       L0044 ERROR| Traceback (most recent call last):
2017-01-13 18:10:08,721 stacktrace       L0044 ERROR|   File "/home/prkumar/github/minishift-b2d-iso/tests/test.py", line 30, in setUp
2017-01-13 18:10:08,721 stacktrace       L0044 ERROR|     self.skip("Skipping testing as no ISO found in 'build' directory.")
2017-01-13 18:10:08,721 stacktrace       L0044 ERROR|   File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 638, in skip
2017-01-13 18:10:08,721 stacktrace       L0044 ERROR|     raise exceptions.TestSkipError(message)
2017-01-13 18:10:08,721 stacktrace       L0044 ERROR| TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,721 stacktrace       L0045 ERROR| 
2017-01-13 18:10:08,721 test             L0593 ERROR| SKIP 1-tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso -> TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,721 test             L0580 INFO | 
2017-01-13 18:10:08,763 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:08,763 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:08,764 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:08,764 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:08,771 multiplexer      L0168 DEBUG| PARAMS (key=timeout, path=*, default=None) => None
2017-01-13 18:10:08,771 test             L0217 INFO | START 2-tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm
2017-01-13 18:10:08,811 test             L0017 INFO | ################################################################
2017-01-13 18:10:08,811 test             L0018 INFO | Avocado version : 43.0
2017-01-13 18:10:08,811 test             L0019 INFO | ################################################################
2017-01-13 18:10:08,811 stacktrace       L0038 ERROR| 
2017-01-13 18:10:08,811 stacktrace       L0041 ERROR| Reproduced traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:436
2017-01-13 18:10:08,812 stacktrace       L0044 ERROR| Traceback (most recent call last):
2017-01-13 18:10:08,812 stacktrace       L0044 ERROR|   File "/home/prkumar/github/minishift-b2d-iso/tests/test.py", line 30, in setUp
2017-01-13 18:10:08,812 stacktrace       L0044 ERROR|     self.skip("Skipping testing as no ISO found in 'build' directory.")
2017-01-13 18:10:08,812 stacktrace       L0044 ERROR|   File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 638, in skip
2017-01-13 18:10:08,812 stacktrace       L0044 ERROR|     raise exceptions.TestSkipError(message)
2017-01-13 18:10:08,812 stacktrace       L0044 ERROR| TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,812 stacktrace       L0045 ERROR| 
2017-01-13 18:10:08,812 test             L0593 ERROR| SKIP 2-tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm -> TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,813 test             L0580 INFO | 
2017-01-13 18:10:08,854 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:08,854 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:08,855 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:08,855 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:08,862 multiplexer      L0168 DEBUG| PARAMS (key=timeout, path=*, default=None) => None
2017-01-13 18:10:08,862 test             L0217 INFO | START 3-tests/test.py:MinishiftISOTest.test_ip_of_vm
2017-01-13 18:10:08,896 test             L0017 INFO | ################################################################
2017-01-13 18:10:08,897 test             L0018 INFO | Avocado version : 43.0
2017-01-13 18:10:08,897 test             L0019 INFO | ################################################################
2017-01-13 18:10:08,897 stacktrace       L0038 ERROR| 
2017-01-13 18:10:08,897 stacktrace       L0041 ERROR| Reproduced traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:436
2017-01-13 18:10:08,897 stacktrace       L0044 ERROR| Traceback (most recent call last):
2017-01-13 18:10:08,897 stacktrace       L0044 ERROR|   File "/home/prkumar/github/minishift-b2d-iso/tests/test.py", line 30, in setUp
2017-01-13 18:10:08,897 stacktrace       L0044 ERROR|     self.skip("Skipping testing as no ISO found in 'build' directory.")
2017-01-13 18:10:08,898 stacktrace       L0044 ERROR|   File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 638, in skip
2017-01-13 18:10:08,898 stacktrace       L0044 ERROR|     raise exceptions.TestSkipError(message)
2017-01-13 18:10:08,898 stacktrace       L0044 ERROR| TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,898 stacktrace       L0045 ERROR| 
2017-01-13 18:10:08,898 test             L0593 ERROR| SKIP 3-tests/test.py:MinishiftISOTest.test_ip_of_vm -> TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,898 test             L0580 INFO | 
2017-01-13 18:10:08,934 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:08,934 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:08,934 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:08,935 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:08,941 multiplexer      L0168 DEBUG| PARAMS (key=timeout, path=*, default=None) => None
2017-01-13 18:10:08,942 test             L0217 INFO | START 4-tests/test.py:MinishiftISOTest.test_docker_env_evaluable
2017-01-13 18:10:08,982 test             L0017 INFO | ################################################################
2017-01-13 18:10:08,982 test             L0018 INFO | Avocado version : 43.0
2017-01-13 18:10:08,982 test             L0019 INFO | ################################################################
2017-01-13 18:10:08,983 stacktrace       L0038 ERROR| 
2017-01-13 18:10:08,983 stacktrace       L0041 ERROR| Reproduced traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:436
2017-01-13 18:10:08,983 stacktrace       L0044 ERROR| Traceback (most recent call last):
2017-01-13 18:10:08,983 stacktrace       L0044 ERROR|   File "/home/prkumar/github/minishift-b2d-iso/tests/test.py", line 30, in setUp
2017-01-13 18:10:08,983 stacktrace       L0044 ERROR|     self.skip("Skipping testing as no ISO found in 'build' directory.")
2017-01-13 18:10:08,983 stacktrace       L0044 ERROR|   File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 638, in skip
2017-01-13 18:10:08,983 stacktrace       L0044 ERROR|     raise exceptions.TestSkipError(message)
2017-01-13 18:10:08,984 stacktrace       L0044 ERROR| TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,984 stacktrace       L0045 ERROR| 
2017-01-13 18:10:08,984 test             L0593 ERROR| SKIP 4-tests/test.py:MinishiftISOTest.test_docker_env_evaluable -> TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:08,984 test             L0580 INFO | 
2017-01-13 18:10:09,036 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:09,036 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:09,036 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:09,036 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:09,044 multiplexer      L0168 DEBUG| PARAMS (key=timeout, path=*, default=None) => None
2017-01-13 18:10:09,044 test             L0217 INFO | START 5-tests/test.py:MinishiftISOTest.test_stopping_vm
2017-01-13 18:10:09,085 test             L0017 INFO | ################################################################
2017-01-13 18:10:09,085 test             L0018 INFO | Avocado version : 43.0
2017-01-13 18:10:09,085 test             L0019 INFO | ################################################################
2017-01-13 18:10:09,086 stacktrace       L0038 ERROR| 
2017-01-13 18:10:09,086 stacktrace       L0041 ERROR| Reproduced traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:436
2017-01-13 18:10:09,086 stacktrace       L0044 ERROR| Traceback (most recent call last):
2017-01-13 18:10:09,086 stacktrace       L0044 ERROR|   File "/home/prkumar/github/minishift-b2d-iso/tests/test.py", line 30, in setUp
2017-01-13 18:10:09,086 stacktrace       L0044 ERROR|     self.skip("Skipping testing as no ISO found in 'build' directory.")
2017-01-13 18:10:09,086 stacktrace       L0044 ERROR|   File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 638, in skip
2017-01-13 18:10:09,086 stacktrace       L0044 ERROR|     raise exceptions.TestSkipError(message)
2017-01-13 18:10:09,087 stacktrace       L0044 ERROR| TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:09,087 stacktrace       L0045 ERROR| 
2017-01-13 18:10:09,087 test             L0593 ERROR| SKIP 5-tests/test.py:MinishiftISOTest.test_stopping_vm -> TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:09,087 test             L0580 INFO | 
2017-01-13 18:10:09,128 sysinfo          L0388 INFO | Commands configured by file: /etc/avocado/sysinfo/commands
2017-01-13 18:10:09,129 sysinfo          L0399 INFO | Files configured by file: /etc/avocado/sysinfo/files
2017-01-13 18:10:09,129 sysinfo          L0419 INFO | Profilers configured by file: /etc/avocado/sysinfo/profilers
2017-01-13 18:10:09,129 sysinfo          L0427 INFO | Profiler disabled
2017-01-13 18:10:09,136 multiplexer      L0168 DEBUG| PARAMS (key=timeout, path=*, default=None) => None
2017-01-13 18:10:09,136 test             L0217 INFO | START 6-tests/test.py:MinishiftISOTest.test_delete_vm
2017-01-13 18:10:09,189 test             L0017 INFO | ################################################################
2017-01-13 18:10:09,189 test             L0018 INFO | Avocado version : 43.0
2017-01-13 18:10:09,190 test             L0019 INFO | ################################################################
2017-01-13 18:10:09,190 stacktrace       L0038 ERROR| 
2017-01-13 18:10:09,190 stacktrace       L0041 ERROR| Reproduced traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:436
2017-01-13 18:10:09,190 stacktrace       L0044 ERROR| Traceback (most recent call last):
2017-01-13 18:10:09,190 stacktrace       L0044 ERROR|   File "/home/prkumar/github/minishift-b2d-iso/tests/test.py", line 30, in setUp
2017-01-13 18:10:09,191 stacktrace       L0044 ERROR|     self.skip("Skipping testing as no ISO found in 'build' directory.")
2017-01-13 18:10:09,191 stacktrace       L0044 ERROR|   File "/usr/lib/python2.7/site-packages/avocado/core/test.py", line 638, in skip
2017-01-13 18:10:09,191 stacktrace       L0044 ERROR|     raise exceptions.TestSkipError(message)
2017-01-13 18:10:09,191 stacktrace       L0044 ERROR| TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:09,191 stacktrace       L0045 ERROR| 
2017-01-13 18:10:09,191 test             L0593 ERROR| SKIP 6-tests/test.py:MinishiftISOTest.test_delete_vm -> TestSkipError: Skipping testing as no ISO found in 'build' directory.
2017-01-13 18:10:09,191 test             L0580 INFO | 
2017-01-13 18:10:09,254 sysinfo          L0107 DEBUG| Not logging /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor (file does not exist)
2017-01-13 18:10:09,271 sysinfo          L0107 DEBUG| Not logging /proc/pci (file does not exist)
2017-01-13 18:10:09,279 sysinfo          L0105 DEBUG| Not logging /proc/slabinfo (lack of permissions)
2017-01-13 18:10:09,284 sysinfo          L0107 DEBUG| Not logging /sys/kernel/debug/sched_features (file does not exist)
2017-01-13 18:10:09,317 job              L0457 INFO | Test results available in /home/prkumar/avocado/job-results/job-2017-01-13T18.10-72d31ce
coolbrg commented 7 years ago

@praveenkumar , It seems you missed on step here before running make test.

You need to run make iso before make test. The tests assume that ISO is already available. See the error below at the end of logs. 

TestSkipError: Skipping testing as no ISO found in 'build' directory.
coolbrg commented 7 years ago

retest this please

coolbrg commented 7 years ago

retest this please

coolbrg commented 7 years ago

retest this please

praveenkumar commented 7 years ago

Tested and worked as expected :+1: 

$ make test
Downloading latest minishift binary...
######################################################################## 100.0%
Done.
avocado run  tests/test.py
JOB ID     : 27c99a0025c7c1b81c95b20821a8666b54a32da1
JOB LOG    : /home/prkumar/avocado/job-results/job-2017-01-17T10.47-27c99a0/job.log
TESTS      : 6
 (1/6) tests/test.py:MinishiftISOTest.test_boot_vm_out_of_iso: PASS (102.61 s)
 (2/6) tests/test.py:MinishiftISOTest.test_ssh_connection_to_vm: PASS (0.16 s)
 (3/6) tests/test.py:MinishiftISOTest.test_ip_of_vm: PASS (0.06 s)
 (4/6) tests/test.py:MinishiftISOTest.test_docker_env_evaluable: PASS (0.07 s)
 (5/6) tests/test.py:MinishiftISOTest.test_stopping_vm: PASS (1.44 s)
 (6/6) tests/test.py:MinishiftISOTest.test_delete_vm: PASS (0.27 s)
RESULTS    : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
TESTS TIME : 104.61 s
coolbrg commented 7 years ago

retest this please

coolbrg commented 7 years ago

retest this please

coolbrg commented 7 years ago

retest this please

coolbrg commented 7 years ago

@hferentschik Tests are passing in ci.centos and ready to merge now.

Once, it is merged will put similar tests for centos-iso repo here https://github.com/minishift/minishift-centos-iso/issues/80

coolbrg commented 7 years ago

So what is the reply to this? For now only CentOS and Linux, right?

Yes @hferentschik

coolbrg commented 7 years ago

I still cannot see which targets would clean the BIN_DIR and hence minishift. It is part of the test install, so there should be a way to clean it as well.

I have added BIN_DIR as part of BUILD_DIR which is being cleaned via clean target. Since iso is also part of build dir, make clean will clean BUILD_DIR and hence BIN_DIR

hferentschik commented 7 years ago

I have added BIN_DIR as part of BUILD_DIR which is being cleaned via clean target. Since iso is also part of build dir, make clean will clean BUILD_DIR and hence BIN_DIR

Got you. +1

hferentschik commented 7 years ago

Merged, thanks