search

voxpupuli / onceover

Your gateway drug to automated infrastructure testing with Puppet
141 stars 45 forks source link

Onceover generated spec directory copies empty files #320

Closed JoshuaSmeda closed 2 years ago

JoshuaSmeda commented 2 years ago

Container: Docker, running under Kubernetes. Using ruby:2.7 image. Onceover release: 3.20.0 Ruby version: ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]

Gem list: >

backticks (1.0.2)
benchmark (default: 0.1.0)
bigdecimal (default: 2.0.0)
blockenspiel (0.5.0)
bundler (default: 2.1.4)
cgi (default: 0.1.0.1)
citrus (3.0.2)
colored (1.2)
colored2 (3.1.2)
concurrent-ruby (1.1.9)
cri (2.15.10)
csv (default: 3.1.2)
date (default: 3.0.3)
dbm (default: 1.1.0)
deep_merge (1.2.1)
delegate (default: 0.1.0)
did_you_mean (default: 1.4.0)
diff-lcs (1.4.4)
etc (default: 1.1.0)
facter (4.2.5)
faraday (0.17.4)
faraday_middleware (0.14.0)
fast_gettext (1.1.2)
fcntl (default: 1.0.0)
fiddle (default: 1.0.0)
fileutils (default: 1.4.1)
forwardable (default: 1.3.1)
gdbm (default: 2.1.0)
getoptlong (default: 0.1.0)
gettext (3.2.9)
gettext-setup (0.34)
git (1.9.1)
hiera (3.7.0)
hiera-eyaml (3.2.2)
highline (2.0.3)
hocon (1.3.1)
httpclient (2.8.3)
io-console (default: 0.5.6)
ipaddr (default: 1.2.2)
irb (default: 1.2.6)
json (default: 2.3.0)
jwt (2.2.3)
little-plugger (1.1.4)
locale (2.1.3)
log4r (1.1.10)
logger (default: 1.4.2)
logging (2.3.0)
matrix (default: 0.2.0)
minitar (0.9)
minitest (5.13.0)
mocha (1.13.0)
multi_json (1.15.0)
multipart-post (2.1.1)
mutex_m (default: 0.1.0)
net-pop (default: 0.1.0)
net-smtp (default: 0.1.0)
net-telnet (0.2.0)
observer (default: 0.1.0)
onceover (3.20.0)
open3 (default: 0.1.0)
openssl (default: 2.1.3)
optimist (3.0.1)
ostruct (default: 0.2.0)
parallel (1.21.0)
parallel_tests (3.7.3)
pastel (0.8.0)
pathspec (1.0.0)
power_assert (1.1.7)
prime (default: 0.1.1)
pstore (default: 0.1.0)
psych (default: 3.1.0)
puppet (6.21.1)
puppet-lint (2.5.2)
puppet-resource_api (1.8.14)
puppet-syntax (3.1.0)
puppet_forge (2.3.4)
puppetlabs_spec_helper (4.0.1)
r10k (3.14.0)
racc (default: 1.4.16)
rake (13.0.6, 13.0.1)
rchardet (1.8.0)
rdoc (default: 6.2.1.1)
readline (default: 0.0.2)
readline-ext (default: 0.1.0)
reline (default: 0.1.5)
rexml (default: 3.2.3.1)
rspec (3.10.0)
rspec-core (3.10.1)
rspec-expectations (3.10.1)
rspec-mocks (3.10.2)
rspec-puppet (2.11.0)
rspec-support (3.10.3)
rspec_junit_formatter (0.4.1)
rss (default: 0.2.8)
sdbm (default: 1.0.0)
semantic_puppet (1.0.4)
semver (1.0.1)
singleton (default: 0.1.0)
stringio (default: 0.1.0)
strscan (default: 1.0.3)
terminal-table (3.0.2)
test-unit (3.3.4)
text (1.3.1)
thor (1.1.0)
timeout (default: 0.1.0)
toml-rb (2.1.0)
tracer (default: 0.1.0)
tty-color (0.6.0)
tty-cursor (0.7.1)
tty-prompt (0.23.1)
tty-reader (0.9.0)
tty-screen (0.8.1)
unicode-display_width (2.1.0)
uri (default: 0.10.0)
versionomy (0.5.0)
webrick (default: 1.6.1)
wisper (2.0.1)
xmlrpc (0.3.0)
yaml (default: 0.1.0)
zlib (default: 1.1.0)

When attempting to run bundle exec onceover run spec, I receive the following error/s:

Upon further investigation, the directory - .onceover/etc/puppetlabs/code/environments/production/spec contains all the necessary files but they are empty when they shouldn't be:

root@runner-hwymybse-project-31-concurrent-0784jv:/builds/HwymyBsE/0/puppet/control/.onceover/etc/puppetlabs/code/environments/production/spec# ls -lsah
total 20K
4.0K drwxr-xr-x 5 root root 4.0K Dec  7 10:49 .
4.0K drwxr-xr-x 9 root root 4.0K Dec  7 10:44 ..
4.0K drwxr-xr-x 2 root root 4.0K Dec  7 10:44 data
4.0K drwxr-xr-x 2 root root 4.0K Dec  7 10:44 factsets
   0 -rw-r--r-- 1 root root    0 Dec  7 10:44 hiera.yaml
   0 -rw-r--r-- 1 root root    0 Dec  7 10:44 onceover.yaml
4.0K drwxr-xr-x 2 root root 4.0K Dec  7 10:44 pre_conditions

These files are not empty under the controlrepo or under .onceover/spec. There is something wrong with the FileUtils.CP.

I've attached the cucumber tests to this issue which has MD5 hashes failures indicating there is a problem with the copying. cucumber_tests.log

It's worth noting that this works locally, in Docker using the same image which indicates that my controlrepo and configuration is likely not the issue but there might be a bug in Onceover.

h4l commented 2 years ago

This is caused by a Linux kernel bug which is affecting Docker. Copying files from a docker-mounted volume to the root filesystem can result in empty files. See here for details: https://github.com/docker/for-linux/issues/1015

In this case it's onceover copying from the repo's files to a dir under /tmp. A workaround using linux is to mount /tmp as tmpfs when running the container, or for Docker on non-linux platforms you can avoid the issue by keeping the temp directory inside the repo's docker volume, e.g. by putting it under ./.onceover/: 

# ruby will create temporary files under the dir specified by the TMP envar
$ export TMP=$(pwd)/.onceover/tmp
# the temp dir has to exist, otherwise Ruby won't use it
$ mkdir -p $TMP
$ bundler exec onceover run
...
dylanratcliffe commented 2 years ago

Wow brilliant catch! I'm going to close this as we have a workaround and it's definitely not a Onceover issue. Hopefully will be resolved upstream soon