search

jantman / vagrant-r10k

UNSUPPORTED - SEEKING MAINTAINER - Vagrant middleware plugin to retrieve puppet modules using r10k.
MIT License
35 stars 12 forks source link

Order-dependent spec failures #10

Closed jantman closed 9 years ago

jantman commented 9 years ago

when rspec is run with --order random, tests fail about 70% of the time. Fix this.

jantman commented 9 years ago

This problem seems to have been introduced by https://github.com/jantman/vagrant-r10k/commit/53c8b267fa23654c5c4ef4c52fba330c1f61bb3c

jantman commented 9 years ago

Using https://github.com/jantman/misc-scripts/blob/master/find_test_order_problems.py I was able to narrow the problem down to somewhere in lines 10-19 of 53c8b26.

jantman commented 9 years ago

adf7c2d fixes this, but introduced one issue with tests using Vagrant < 1.7.2 (should be fixed in b8b053f) and shows code coverage of plugin.rb decreasing from 89% to 22.22% despite adding tests.

jantman commented 9 years ago

Confirmed that without plugin_spec.rb, coverage of plugin.rb shows as 88.89%, missing only lines 4 and 8. With that spec, it shows as 22.22%, covering only lines 1-2 and 7-8. This honestly makes no sense to me at all. The problem seems to be lines 10-15 in that spec, the 'unsupported vagrant version' context that re-loads the plugin code.

jantman commented 9 years ago

I'm going to just say that (a) loading the plugin outside of vagrant itself, or (b) using the plugin with Vagrant < 1.2, are very unlikely edge cases. Removing tests for them, and ignoring from coverage.

jantman commented 9 years ago

coverage stuff fixed in ad4635a