dpc10ster / RJafroc

Artificial Intelligence: Evaluating AI, optimizing AI
19 stars 8 forks source link

Fix structure differences across R versions in StSignificanceTesting.R #23

Closed pwep closed 5 years ago

pwep commented 5 years ago

A bug labelled Type inconsistency when converting arrays to data frames was fixed in R v3.6.1

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17570

The fix changes some simple array structure to a vector when included in a data.frame. Older versions <3.6.1 did not do this as expected. This appears to be the source of the errors in the test of StSignificanceTesting.R. Do have a look at the simple example code on the bug fix page for more detail.

This pull request adds a as.vector call to the relevant parts of the output in StSignificanceTesting.R - it should not have any impact for v3.6.1, but bring earlier versions into line. I have added it to StSignificanceTesting.R so sensible output is generated, rather than try to catch the individual issues in the test itself.

Travis should pass on all three versions (versions currently: oldrel: 3.5.3; release: 3.6.0; devel: is a recent pull from R dev tree which will become R 3.6.1). The goodValues (not changed in this pull request) have all been generated previously in R 3.6.1.

codecov[bot] commented 5 years ago

Codecov Report

Merging #23 into development will not change coverage. The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff              @@
##           development      #23   +/-   ##
============================================
  Coverage        64.74%   64.74%           
============================================
  Files               56       56           
  Lines             7457     7457           
============================================
  Hits              4828     4828           
  Misses            2629     2629
Impacted Files Coverage Δ
R/StSignificanceTesting.R 72.93% <100%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update cd2e0b2...b9c8d2f. Read the comment docs.

dpc10ster commented 5 years ago

Worked! Thanks Peter; I will check other files where this type of change might be appropriate; I must say the student's coding held up! It turned out to be a R bug - I would never have figured that.

dpc10ster commented 5 years ago

Still having problems with test

Incorporated Peter proposed changes (as.vector()) to StdError and DF in StSignificanceTestingCrossedModalities.R; did not alter StSingleFixedFactor.R

Changed goodValues directory to goodValues361, meaning the values are for R version 3.6.1

Regenerated good values on laptop, which is running 3.6.0;

Moved goodvalues for SigTest and SigTestCad from values from Peter's FTP website to RJafroc testthat directory; after adding .rds extension to all of the files

devtools::test() ran with no errors

But R CMD check fails:

── Test failures ───────────────────────────────────────────────── testthat ────

library("testthat") library("RJafroc")

test_check("RJafroc") ── 1. Failure: SignificanceTestingAllCombinations (@test-significance-testing CurrentValues[[listMem]] not equal to GoodValues[[listMem]]. Component "Source": Attributes: < Component "levels": 3 string mismatches > List member = 2, Dataset = dataset02, FOM = Wilcoxon, method = DBMH

── 2. Failure: SignificanceTestingAllCombinations (@test-significance-testing CurrentValues[[listMem]] not equal to GoodValues[[listMem]]. Component "Source": Attributes: < Component "levels": 3 string mismatches > List member = 2, Dataset = dataset05, FOM = HrAuc, method = DBMH

══ testthat results ════════════════════════════════════════════════════════ OK: 283 SKIPPED: 0 WARNINGS: 0 FAILED: 2

  1. Failure: SignificanceTestingAllCombinations (@test-significance-testing.R#47)
  2. Failure: SignificanceTestingAllCombinations (@test-significance-testing.R#47)
dpc10ster commented 5 years ago

Need help Peter! These look identical to me, yet it fails R CMD check (but passes dev tools::test) and passes all Travis checks; strange

GoodValues <- readRDS(fn) CurrentValues <- StSignificanceTesting(dataset, FOM = FOM_arr[i],method = method_arr[j]) str(GoodValues[[2]]) 'data.frame': 8 obs. of 4 variables: $ Source: Factor w/ 8 levels "C","R","RC","T",..: 4 2 1 7 5 3 8 6 $ SS : num 0.547 1.749 44.846 0.251 11.283 ... $ DF : num 1 4 113 4 113 ... $ MS : num 0.5468 0.4373 0.3969 0.0628 0.0998 ... str(CurrentValues[[2]]) 'data.frame': 8 obs. of 4 variables: $ Source: Factor w/ 8 levels "C","R","RC","T",..: 4 2 1 7 5 3 8 6 $ SS : num 0.547 1.749 44.846 0.251 11.283 ... $ DF : num 1 4 113 4 113 ... $ MS : num 0.5468 0.4373 0.3969 0.0628 0.0998 ... attributes(GoodValues[[2]]) $names [1] "Source" "SS" "DF" "MS"

$class [1] "data.frame"

$row.names [1] 1 2 3 4 5 6 7 8

attributes(CurrentValues[[2]]) $names [1] "Source" "SS" "DF" "MS"

$class [1] "data.frame"

$row.names [1] 1 2 3 4 5 6 7 8