Solver Inconsistency/Unavailability?

[adamrauh]

I'm making this thread to document some problems I've been having with reproducing results consistently + some other error messages related to the solver and hopefully resolve them.

I noticed that when I was running identical code with identical inputs multiple times, I would sometimes get a solution (a solution was expected) and other times it would return no solution.

Back during the process of setting up and building the package for the first time, I had some problems/errors related to gfortran. After talking with @benthestatistician it sounds like others have had some issues with this as well, so some combination of gfortran (I have 4.8.2) + R (3.3.2) + OS (10.12) versions may be the cause of the random inconsistency problem.

Using another machine (hopefully as a workaround), I had no issues building the package with gfortran 4.4.7 already installed on it, but when I ran some tests to see if the consistency issue persisted, I got an error saying, "relaxalg not available for .Fortran() for package optmatch." After some digging, I believe this problem is related to not properly loading/connecting the subroutine in relax4s.f to R. According to this link I need to create a .dll and then use dyn.load(...) to connect things in R. I have to play with that a bit more, but I'm sure there's a better solution than having to go through that process every time.

I've attached the test I use to check the reproducibility problem, which I'm also using to currently trigger the error about relaxalg not being available as well.

Any ideas on either of these would be excellent.

consistencytest.Rmd.zip

[benthestatistician]

1. I think I know what the other machine is, "evalengin"; if so it's running RHEL 6.9. (Got this via $ cat /etc/redhat-release.) Please correct this if it's mistaken.
2. If I'm not mistaken, the dll advice is specific to MS Windows. (The linked wiki page doesn't directly speak to this, but lower on the page there's a path spec starting w/ "d:/", consistent with MS Windows but not *nix.)
3. I thought you gotten past the "relaxalg not available" problem. Is this not the case?
4. Specifically, having built the package can you successfully execute the following, from within R:

     > setwd("<top level dir of optmatch clone>")
     > library("devtools")
     > test()

[adamrauh]

1. RHEL 6.9 is correct
2. Yes, you are also correct here -- my mistake. I think I found the *nix "equivalent" here. I tried this with relax4s.f and had no luck, unfortunately. Calling is.loaded("relaxalg") returns TRUE now instead of FALSE, but it didn't stop the error. There's a chance I could be out in left field thinking along these lines.
3. Unfortunately no. Although to be clear, I don't have the "relaxalg not available" problem on my own machine, just the inability to reproduce results consistently. The evalengin machine is the one with the relaxalg issue. I've been taking unsuccessful stabs at resolving the evalengin problem most recently. I could try downgrading gfortran on my machine to 4.4.7 and see if that changes anything regarding the inconsistencies.
4. I attached the output of test(). I am seeing quite a bit of the same error show up repeatedly. test()output.txt

[benthestatistician]

Odd. Did you precisely follow the steps I suggested in comments to #127, @adamrauh ? Did you try them again?

I got onto the evalengin machine and tried those steps again. Specifically:

$ cd <optmatch clone directory>
$ R
> remove.packages("optmatch") 
> q('no')
$ make clean
$ R
 > devtools::install()
 > devtools::test()

...and it worked for me. (There were a few questions along the way, about whether it should install this or that.)

I'd recommend that you check to see whether this can be done in your account with the master branch of optmatch, cloned directly from the original (markmfredrickson) repo. @timlycurgus also has an account on evalengin, so if you can't get this to work you can ask him to try it: that will tell us whether there's something special about my account that makes it work from it, or whether there's something special about your account that makes it not work.

Conceivably it has to do with what R packages are already installed. Here's what I see in my per-user package directory:

[bbh@evalengin optmatch]$ ls ~/R/x86_64-pc-linux-gnu-library/3.3/
abind       brew        devtools  jsonlite  pkgconfig     Rcpp        SparseM   tidyr
assertthat  brglm       digest    magrittr  plogr         RItools     stringi   tidyselect
backports   commonmark  dplyr     memoise   praise        rlang       stringr   whisker
BH          crayon      git2r     mime      profileModel  roxygen2    svd       withr
bindr       curl        glue      openssl   purrr         rprojroot   testthat  xml2
bindrcpp    desc        httr      optmatch  R6            rstudioapi  tibble    xtable

[adamrauh]

Ok, after trying the above a few different ways, the issue looks to be related to my own fork of the hinting branch. I tried the steps above with the master branch, the hinting branch from this repo, and my own fork-- the first two worked fine, but my fork still had the same issues. Additionally, using the two working hinting/master branches, I was able to run my own tests about getting consistent results, and everything actually came back perfectly, which is good news.

I think the fastest route to getting operational would be to make a new fork of the existing hinting branch on this repo, then re-make the previously made changes about reduced costs in the hopes of either getting a new working branch, or discovering which particular change is causing the problem.

[benthestatistician]

@adamrauh, I suspect this may be ready for closing. Do you agree?

[adamrauh]

@benthestatistician Yep, this can definitely be closed.