ravingmantis / unittest

Unit testing for R
4 stars 1 forks source link

Test failure on r-devel-windows-x86_64-new-UL #3

Closed lentinj closed 2 years ago

lentinj commented 2 years ago

There's currently a test failure: https://cran.r-project.org/web/checks/check_results_unittest.html

 not ok - Environments get converted to lists
 # Test returned non-TRUE value:
 # Length mismatch: comparison on first 2 components
 # Component "a": Mean relative difference: 0.6666667
 # <1b>[1m--- as.environment(list(a = 3, b = 4))<1b>[m
 # <1b>[1m+++ as.environment(list(a = 5, b = 4, c = 9))<1b>[m
 # <1b>[32m{+$c
 +}<1b>[m
 # <1b>[32m{+[1] 9+}<1b>[m
 #
 # $b<1b>[m
 # [1] 4<1b>[m
 #
 # $a<1b>[m
 # [1] <1b>[31m[-3-]<1b>[m<1b>[32m{+5+}<1b>[m

...without escapes (which should be filtered before comparsion) this looks like:

# Test returned non-TRUE value:
# Length mismatch: comparison on first 2 components
# Component "a": Mean relative difference: 0.6666667
# --- as.environment(list(a = 3, b = 4))
# +++ as.environment(list(a = 5, b = 4, c = 9))
# {+$c+}
# {+[1] 9+}
#
# $b
# [1] 4
#
# $a
# [1] [-3-]{+5+}

The current version on CRAN will show the raw output from the test that failed. The only difference to the expected output is the carriage return after $c, but I have no idea where it comes from.

No way of replicating this, other than re-releasing and seeing if it fails again.

lentinj commented 2 years ago

Current timings are 8.00 | 78.00 | 86.00, to see if they change (i.e. the test has re-run).

lentinj commented 2 years ago

Despite supposedly being split by newline, the offending item in the vector has an errant "\r" in it:

chr [1:12] "Length mismatch: comparison on first 2 components" "Component \"a\": Mean relative difference: 0.6666667" "--- as.environment(list(a = 3, b = 4))" "+++ as.environment(list(a = 5, b = 4, c = 9))" "{+$c\r+}" . . .
lentinj commented 2 years ago

The above is a bit of a cheat, but OTOH it's not particularly dangerous, given the output should be split by newline anyway.

lentinj commented 2 years ago

CRAN checks are happy, so we're happy.