dpc10ster / RJafroc

Artificial Intelligence: Evaluating AI, optimizing AI
19 stars 8 forks source link

Suppress warnings generated by first runs of expect_known_output in tests #10

Closed pwep closed 5 years ago

pwep commented 5 years ago

The expect_known_output function produces a warning the first time a file is created. This is expected behaviour but fills the testthat output with multiple (and redundant) warnings. It should be possible to wrap the first call of expect_known_output with expect_warning.

Example:

test_that("DfBinDatasetAFROC", {
  tmp <- tempfile()

  expect_warning(    # Expect a warning to be generated...
    expect_known_output(
      DfBinDataset(dataset05, opChType = "AFROC"), 
      tmp, print = TRUE),
  "Creating reference output")  # ...with a message matching this string

  expect_known_output(
    DfBinDataset(dataset05, opChType = "AFROC"), 
    tmp, print = TRUE)
})
pwep commented 5 years ago

I have added a wrapper around all first instances of expect_known_output within all tests. All tests now appear to pass (without flagging the Create reference output warning.) (See 40cf6c6 - will create a pull request if this approach is appropriate)

> devtools::test()
Loading RJafroc
Testing RJafroc
✔ |  OK F W S | Context
✔ |   3       | compare-3fits [119.7 s]
✔ |   6       | corcbm [20.7 s]
✔ |  76       | data-file [102.6 s]
✔ |  28       | test datasets [10.0 s]
✔ |  12       | Fitting routines [799.7 s]
✔ |  35       | UtilFigureOfMerit tests [1.4 s]
✔ |   8       | Significance testing excluding CAD [7.0 s]
✔ |  12       | Output report [19.5 s]
✔ |  18       | Plotting routines [30.5 s]
✔ |  10       | Sample Size ROC [20.9 s]
✔ |   8       | ROC sample size routines [11.9 s]
✔ |  45       | Significance testing excluding CAD [54.0 s]
✔ |   6       | Simulate data sets [1.7 s]
✔ |  30       | Significance testing: StSignificanceTestingCadVsRadiologists [182.6 s]
✔ |  27       | utils [5.6 s]

(I used a Pentium-based Chromebook so my test times are very slow)

dpc10ster commented 5 years ago

I have added a wrapper around all first instances of expect_known_output within all tests. All tests now appear to pass (without flagging the Create reference output warning.) (See 40cf6c6 - will create a pull request if this approach is appropriate)

> devtools::test()
Loading RJafroc
Testing RJafroc
✔ |  OK F W S | Context
✔ |   3       | compare-3fits [119.7 s]
✔ |   6       | corcbm [20.7 s]
✔ |  76       | data-file [102.6 s]
✔ |  28       | test datasets [10.0 s]
✔ |  12       | Fitting routines [799.7 s]
✔ |  35       | UtilFigureOfMerit tests [1.4 s]
✔ |   8       | Significance testing excluding CAD [7.0 s]
✔ |  12       | Output report [19.5 s]
✔ |  18       | Plotting routines [30.5 s]
✔ |  10       | Sample Size ROC [20.9 s]
✔ |   8       | ROC sample size routines [11.9 s]
✔ |  45       | Significance testing excluding CAD [54.0 s]
✔ |   6       | Simulate data sets [1.7 s]
✔ |  30       | Significance testing: StSignificanceTestingCadVsRadiologists [182.6 s]
✔ |  27       | utils [5.6 s]

(I used a Pentium-based Chromebook so my test times are very slow)

Good idea; please generate the pull request

dpc10ster commented 5 years ago

I am going to close this issue pending your pull request so that I can incorporate changes into development branch

dpc10ster commented 5 years ago

Thanks; am closing this issue; your changes have been merged and the file test_FOM has been cleaned up; am still remaining on other tests