cvxgrp / CVXR

An R modeling language for convex optimization problems.
https://cvxr.rbind.io/
Apache License 2.0
202 stars 32 forks source link

many of the unit tests are failing #56

Closed flodel closed 5 years ago

flodel commented 5 years ago

I am relying enough on this package (thank you so much for your work) that I would like to help. I have forked from master and was planning on working on these couple issues (https://github.com/cvxgrp/CVXR/issues/55 and https://github.com/cvxgrp/CVXR/issues/31) to eventually submit pull requests.

In the process, I wanted to run the unit tests to add my own tests, but also make sure that my changes would not break current behaviors. Unfortunately, I see that a lot of the unit tests are failing (one example below) and I can't tell if it is because of my specific environment, or if it is indeed the case that the unit tests need some rework.

Also, I note that there are a few print() and calls to solver(..., verbose = TRUE) within the tests, which make running them in interactive mode particularly difficult to read.

I would hope that the master branch consistently pass all of its tests and create a clean output. Thank you if you can address.


Example output, after running devtools::test(filter = "g01-atoms")

Loading CVXR
Creating a generic function for ‘is.element’ from package ‘base’ in package ‘CVXR’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘CVXR’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘CVXR’
Creating a generic function for ‘%x%’ from package ‘base’ in package ‘CVXR’
Testing CVXR
✔ |  OK F W S | Context
✖ | 263 2   9 | g01-atoms [3.7 s]
────────────────────────────────────────────────────────────────────────────────
test-g01-atoms.R:291: error: test the vstack function
object of type 'S4' is not subsettable
1: .handleSimpleError(function (e) 
   {
       handled <<- TRUE
       test_error <<- e
       options(expressions = expressions_opt_new)
       on.exit(options(expressions = expressions_opt), add = TRUE)
       e$expectation_calls <- frame_calls(11, 2)
       test_error <<- e
       register_expectation(e)
       e$handled <- TRUE
       test_error <<- e
   }, "object of type 'S4' is not subsettable", quote(x[i, j])) at /Users/flodel/git/CVXR0/tests/testthat/test-g01-atoms.R:291
2: eval(code, test_env)

test-g01-atoms.R:449: error: test DCP properties of partial optimize
object of type 'S4' is not subsettable
1: Problem(Maximize(square(t[1])), list(-t <= x, x <= t)) at /Users/flodel/git/CVXR0/tests/testthat/test-g01-atoms.R:449
2: .Problem(objective = objective, constraints = constraints) at /Users/flodel/git/CVXR0/R/problem.R:304
3: new(structure("Problem", package = "CVXR"), ...)
4: initialize(value, ...)
5: initialize(value, ...)
6: .local(.Object, ...)
7: Maximize(square(t[1])) at /Users/flodel/git/CVXR0/R/problem.R:314
8: .Maximize(expr = expr) at /Users/flodel/git/CVXR0/R/problem.R:73
9: new(structure("Maximize", package = "CVXR"), ...)
10: initialize(value, ...)
11: initialize(value, ...)
12: .local(.Object, ...)
13: as.Constant(expr) at /Users/flodel/git/CVXR0/R/problem.R:20
14: is(expr, "Expression") at /Users/flodel/git/CVXR0/R/constant.R:104
15: square(t[1])
16: .Square(args = list(x)) at /Users/flodel/git/CVXR0/R/elementwise.R:1100
17: new(structure("Square", package = "CVXR"), ...)
18: initialize(value, ...)
19: initialize(value, ...)
20: .local(.Object, ...)
21: lapply(args, function(arg) {
       as.Constant(arg)
   }) at /Users/flodel/git/CVXR0/R/atoms.R:17

test-g01-atoms.R:455: skip: test the partial_optimize eval 1-norm
Reason: empty test

test-g01-atoms.R:490: skip: test partial_optimize min 1-norm
Reason: empty test

test-g01-atoms.R:506: skip: test partial_optimize simple problem
Reason: empty test

test-g01-atoms.R:522: skip: test partial_optimize special var
Reason: empty test

test-g01-atoms.R:538: skip: test partial_optimize special constr
Reason: empty test

test-g01-atoms.R:554: skip: test partial_optimize with parameters
Reason: empty test

test-g01-atoms.R:572: skip: test partial_optimize numeric function
Reason: empty test

test-g01-atoms.R:605: skip: test partial_optimize stacked
Reason: empty test

test-g01-atoms.R:622: skip: test the NonNegative Variable class
Reason: empty test
────────────────────────────────────────────────────────────────────────────────

══ Results ═════════════════════════════════════════════════════════════════════
Duration: 3.9 s

OK:       263
Failed:   2
Warnings: 0
Skipped:  9

If it is useful, here is my sessionInfo():

R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] CVXR_0.99-6     testthat_2.2.1  devtools_1.12.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.2        xml2_1.1.1        magrittr_1.5      roxygen2_6.0.1   
 [5] bit_1.1-12        lattice_0.20-35   R6_2.2.2          rlang_0.4.0      
 [9] stringr_1.2.0     tools_3.4.1       grid_3.4.1        R.oo_1.22.0      
[13] cli_1.1.0         withr_2.1.2       scs_1.2-3         commonmark_1.2   
[17] assertthat_0.2.0  bit64_0.9-7       digest_0.6.12     crayon_1.3.4     
[21] Matrix_1.2-10     gmp_0.5-13.1      ECOSolveR_0.5.2   codetools_0.2-15 
[25] R.utils_2.9.0     memoise_1.1.0     stringi_1.1.5     compiler_3.4.1   
[29] Rmpfr_0.7-1       R.methodsS3_1.7.1
bnaras commented 5 years ago

The tests all pass as intended; the package would not be on CRAN if the tests on the master branch did not pass. See below.

> test_check("CVXR", filter="^g01")
══ testthat results  ═══════════════════════════════════════════════════════════
[ OK: 264 | SKIPPED: 10 | WARNINGS: 0 | FAILED: 0 ]

What is true, however, is that we could clean up (e.g. skipped tests) and improve the tests to make them Rstudio-friendly and provide better messages. CVXR is a complex package and we welcome your participation, but please note that most action is happening on the 1.0 branch. This is significantly different from 0.99-x and is very actively being developed.