CDLUC3 / stash

A UC3 application framework for storing and sharing research data
MIT License
6 stars 4 forks source link

Intermittent test failures due to strict time matching #7

Open dmolesUC opened 6 years ago

dmolesUC commented 6 years ago

Several subprojects have a custom :be_time RSpec matcher for comparing times. Currently this is string-based and compares down to 1/100 second. This is generally loose enough on the desktop but causes intermittent failures on Travis, e.g. this one in build 581:

  1) StashEngine::Resource update_publication_date! falls back to the current time
     Failure/Error: expect(resource.publication_date).to be_time(Time.now)

       expected time:
       2018-11-14T17:55:17.98Z

       but was:
       2018-11-14T17:55:17.97Z
     # ./spec/db/stash_engine/resource_spec.rb:172:in `block (3 levels) in <module:StashEngine>'
Finished in 6.71 seconds (files took 1.51 seconds to load)
247 examples, 1 failure

A fix would be to replace the string-based matcher with one based on (1) timezone offset and (2) fractional seconds, with a sloppier delta, such as the one used in mrt-dashboard, which only checks accuracy within a half-second either way.

(Note though that the mrt-dashboard matcher is only used by a couple of tests, and using it globally in stash might reveal some bugs.)

For stash, this would have to be substituted in five files:

stash-wrapper/spec/rspec_custom_matchers.rb
stash-merritt/spec/rspec_custom_matchers.rb
stash_engine/spec/rspec_custom_matchers.rb
stash_datacite/spec/rspec_custom_matchers.rb
stash-harvester/spec/rspec_custom_matchers.rb

(At some point it might be worth pulling out the custom matchers into a shared file, either as a standalone gem or just a directory with symlinks. Note that they're not all quite the same and some may be newer or more fully-featured than others.)