puppetlabs / puppetlabs-wsus_client

puppetlabs - wsus_client module
Apache License 2.0
10 stars 32 forks source link

Integer test cases in init_spec.rb are not strong enough #182

Closed LukasAud closed 2 years ago

LukasAud commented 2 years ago

Several test cases in the init_spec.rb file are not up to standards. The problem lies in the reusable methods (shared_examples) defined at the start of the file (lines 52, 61 and 224), where the String message that is used to check against the expected error (in expected failures) is not complete. This is due to a problem encountered during the development of https://github.com/puppetlabs/puppetlabs-wsus_client/pull/181 where attempting to create a String that resembles the part of the error message would result in a check failure despite the error message containing the exact String defined.

Expected Behavior

The test cases should have a comprehensive String that properly checks for the correct error message instead of the current, very limited String that is established. The test file run should pass with no failures.

Steps to Reproduce the problem

Steps to reproduce the behavior:

  1. Pull down the latest version of the module and check that there are no errors (running 'bundle exec rake parallel_spec')
  2. Go to init_spec.rb and change the expected error message in any of the following lines > 54, 63, 237 to a random String
  3. Run 'bundle exec rake parallel_spec' a take note of the error message being thrown
  4. Change the String again to contain an exact copy the entire error message (or at least the part containing an array [x, y])
  5. Test now should still fail despite both messages containing the same data character by character

Environment

Additional Context

The exact problem seems to be the comparison of arrays within the error messages against a String in the expected message. For example:

if the error message was: "PuppetError: Failed validation: parameter X is expecting a value of undef or Integer[2, 15], got [1, 1]" then the String "parameter X is expecting a value of undef or Integer" would work and pass the test but the String "parameter X is expecting a value of undef or Integer[2, 15]" would not work.

pmcmaw commented 2 years ago

Closing in favor of tracking in JIRA: https://tickets.puppetlabs.com/browse/CONT-192