[Test Failure] R: CPU - Githubissues

Chancebair commented 5 years ago

Build failure error in http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/master/2038/pipeline

** testing if installed package can be loaded
* DONE (mxnet)
+ make rpkgtest R_LIBS=/tmp/r-site-library
Rscript -e 'require(testthat);res<-test_dir("R-package/tests/testthat");if(!testthat:::all_passed(res)){stop("Test failures", call. = FALSE)}'
Loading required package: testthat
v | OK F W S | Context
trying URL 'https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/R/data/ISBI.zip'
Content type 'application/zip' length 25275513 bytes (24.1 MB)
==================================================
downloaded 24.1 MB

/ |  0       | Image segmentation
- |  0 1     | Image segmentation
x |  0 1     | Image segmentation [2.5 s]
--------------------------------------------------------------------------------
test_img_seg.R:157: error: UNET
could not find function "mx.ctx.default"
1: .handleSimpleError(function (e) 
   {
       handled <<- TRUE
       test_error <<- e
       options(expressions = expressions_opt_new)
       on.exit(options(expressions = expressions_opt), add = TRUE)
       e$expectation_calls <- frame_calls(11, 2)
       test_error <<- e
       register_expectation(e)
       e$handled <- TRUE
       test_error <<- e
   }, "could not find function \"mx.ctx.default\"", quote(mx.ctx.default())) at R-package/tests/testthat/test_img_seg.R:157

vrakesh commented 5 years ago

@Chancebair Thank you for reporting the build failure.

vrakesh commented 5 years ago

@mxnet-label-bot add [Build]

harshp8l commented 5 years ago

@mxnet-label-bot add [build]

vandanavk commented 5 years ago

@anirudhacharya @ankkhedia

anirudhacharya commented 5 years ago

this is not a build failure. it is a flaky test failure

srochel commented 5 years ago

@anirudhacharya - please elaborate why you think "}, "could not find function \"mx.ctx.default\"", quote(mx.ctx.default())) at R-package/tests/testthat/test_img_seg.R:157" is a flaky test?

anirudhacharya commented 5 years ago

@srochel Still investigating the root cause, but it is a test failure because it failed in the make rpkgtest step which is run after the make rpkg( which builds the package) is run successfully. And it is flaky because it is not consistently reproducible.

But unlike most flaky tests it does not seem to be a precision issue with output values, but I still need to root cause why the interpreter is not able to find the function mx.ctx.default.

When I tried to replicate I was able to successfully run the test_img_seg.R( the test that is erroring out in the above issue) test locally and since this failure there have also been successful runs on the CI - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-13467/2/pipeline

harshp8l commented 5 years ago

@mxnet-label-bot update [flaky, test, r]

anirudhacharya commented 5 years ago

@mxnet-label-bot add [CI]

ChaiBapchya commented 4 years ago

@anirudhacharya R GPU failed in unix-gpu for unrelated PR #17542

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-17542/10/pipeline/

R-package/Makefile:32: recipe for target 'rpkgtest' failed

Not sure exactly which test failed. Any idea? Looks like flaky coz other unix-gpu builds seem to have gone fine.

apache / mxnet

[Test Failure] R: CPU #13439