search

voxpupuli / puppet-virtualbox

A Puppet module to install and manage VirtualBox
https://forge.puppet.com/puppet/virtualbox
Apache License 2.0
5 stars 30 forks source link

Update acceptance tests and disable them on GitHub Actions #132

Closed smortex closed 1 year ago

smortex commented 1 year ago

The test suite use versions of VirtualBox which have been EOL for a long time. Packages for these old VirtualBox versions on the modern operating systems this module support are not available.

Rework the test suite to use current VirtualBox versions on current Operating Systems. Also disable acceptance tests on GitHub actions because docker does not allow us to test them reliably.

This PR also include:

smortex commented 1 year ago

Meh, GitHub action rely on Docker so I don't think we can run the acceptance tests this way: the kernel detected in each System Under Test (SUT) environment is the one of the host system, and searching for the corresponding headers in the SUT is likely to fail, e.g.

E: Couldn't find any package by regex 'linux-headers-5.15.0-1020-azure'
Error: /Stage[main]/Virtualbox::Kernel/Package[linux-headers-5.15.0-1020-azure]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install linux-headers-5.15.0-1020-azure' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package linux-headers-5.15.0-1020-azure
E: Couldn't find any package by glob 'linux-headers-5.15.0-1020-azure'
E: Couldn't find any package by regex 'linux-headers-5.15.0-1020-azure'

Running the acceptance tests with BEAKER_HYPERVISOR=vagrant BEAKER_setfile=ubuntu1804-64 bundle exec rake beaker on my FreeBSD box pass. There seems to be some GitHub action to run command in Vagrant images that maybe can be used to workaround the issue but that seems to require osx images to run and looks very fragile IMHO.

Should we just disable acceptance tests for this module?

ekohl commented 1 year ago

Should we just disable acceptance tests for this module?

I think so - they were disabled prior to this change so I'd say it's ok to keep them disabled.

smortex commented 1 year ago

I opened https://github.com/voxpupuli/modulesync_config/pull/798 to add a flag to handle this nicely.

I am checking each config 1 by 1 on my local machine to ensure we have acceptance tests that works and avoid them rotting even further, then I'll push the config to disable them.