Multiple webmock tests, but not all, were randomly failing on CCI only. These were first seen when PR #259 was merged into master, but were not caused by the PR. They appeared to be related to dependency changes or infrastructure change at CCI.
After extensive investigation (many thanks to @botimer and @jrgriffiniii), it was determined that the addition of rest-client gem caused an error to surface. The error occurs because when rest-client gem is present, RDF::Graph.load method returns its error messages in a different form. Sometimes the error message includes parentheses around the error code (i.e., "... (503)") and sometimes not (i.e., "... 503"). QA parses the error code from the error message to pass back a more meaningful error to the caller.
Solution
QA was doing a positional parse expecting the message to be consistent. It was updated to use regex which is less fragile.
A Better Solution
Ultimately, the fix would be for RDF::Graph.load to...
1) minimally return the error message in the same format every time
2) improve exceptions by throwing specific exceptions based on the error code that can be captured in the calling application
Problem
Multiple webmock tests, but not all, were randomly failing on CCI only. These were first seen when PR #259 was merged into master, but were not caused by the PR. They appeared to be related to dependency changes or infrastructure change at CCI.
After extensive investigation (many thanks to @botimer and @jrgriffiniii), it was determined that the addition of rest-client gem caused an error to surface. The error occurs because when rest-client gem is present, RDF::Graph.load method returns its error messages in a different form. Sometimes the error message includes parentheses around the error code (i.e., "... (503)") and sometimes not (i.e., "... 503"). QA parses the error code from the error message to pass back a more meaningful error to the caller.
Solution
QA was doing a positional parse expecting the message to be consistent. It was updated to use regex which is less fragile.
A Better Solution
Ultimately, the fix would be for RDF::Graph.load to... 1) minimally return the error message in the same format every time 2) improve exceptions by throwing specific exceptions based on the error code that can be captured in the calling application
Related Issues
Issue #260 - Periodically Failing Tests resulting in failing master build
https://github.com/samvera/maintenance/issues/27 Investigate webmock dependencies