MLopez-Ibanez / irace

Iterated Racing for Automatic Algorithm Configuration
https://mlopez-ibanez.github.io/irace/
GNU General Public License v2.0
58 stars 14 forks source link

Error: == irace == all(colAnys(!is.na(elite.data))) is not TRUE #57

Closed MLopez-Ibanez closed 1 year ago

MLopez-Ibanez commented 1 year ago

test-capping.R:76:3 fails intermittently and it is not clear why. The assert should be correct but something is breaking this assumption.

── Error ('test-capping.R:76:3'): cap.irace targetRunner = target.runner.reject, maxTime = 10000 ──
Error: Error: == irace == all(colAnys(!is.na(elite.data))) is not TRUE

42: elitist_race(scenario = scenario, configurations = raceConfigurations, 
        parameters = parameters, maxExp = currentBudget, minSurvival = minSurvival, 
        elite.data = elite.data, elitistNewInstances = if (firstRace) 0L else scenario$elitistNewInstances, 
        full_experiment_log = iraceResults$experimentLog) at irace.R#1264
41: irace_run(scenario = scenario, parameters = parameters) at irace.R#710
40: irace_common(scenario, parameters, simple = TRUE) at irace.R#682
39: irace(scenario = scenario, parameters = parameters) at test-capping.R#57
38: cap.irace(targetRunner = target.runner.reject, maxTime = 10000, 
        debugLevel = 3) at test-capping.R#76
37: eval(code, test_env)
36: eval(code, test_env)
35: withCallingHandlers({
        eval(code, test_env)
        if (!handled && !is.null(test)) {
            skip_empty()
        }
    }, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, 
        message = handle_message, error = handle_error)
34: doTryCatch(return(expr), name, parentenv, handler)
33: tryCatchOne(expr, names, parentenv, handlers[[1L]])
32: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
31: doTryCatch(return(expr), name, parentenv, handler)
30: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), 
        names[nh], parentenv, handlers[[nh]])
29: tryCatchList(expr, classes, parentenv, handlers)
28: tryCatch(withCallingHandlers({
        eval(code, test_env)
        if (!handled && !is.null(test)) {
            skip_empty()
        }
    }, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, 
        message = handle_message, error = handle_error), error = handle_fatal, 
        skip = function(e) {
        })
27: test_code(desc, code, env = parent.frame(), reporter = reporter)
26: test_that("cap.irace targetRunner = target.runner.reject, maxTime = 10000", 
        {
            skip_on_cran()
            generate.set.seed()
            cap.irace(targetRunner = target.runner.reject, maxTime = 10000, 
                debugLevel = 3)
        }) at test-capping.R#73
25: force(code)
24: withr::with_output_sink("test-capping.Rout", {
        target.runner <- function(experiment, scenario) {
            debugLevel <- scenario$debugLevel
            configuration.id <- experiment$id.configuration
            instance.id <- experiment$id.instance
            seed <- experiment$seed
            configuration <- experiment$configuration
23: eval(code, test_env)
22: eval(code, test_env)
21: withCallingHandlers({
        eval(code, test_env)
        if (!handled && !is.null(test)) {
            skip_empty()
        }
    }, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, 
        message = handle_message, error = handle_error)
20: doTryCatch(return(expr), name, parentenv, handler)
19: tryCatchOne(expr, names, parentenv, handlers[[1L]])
18: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
17: doTryCatch(return(expr), name, parentenv, handler)
16: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), 
        names[nh], parentenv, handlers[[nh]])
15: tryCatchList(expr, classes, parentenv, handlers)
14: tryCatch(withCallingHandlers({
        eval(code, test_env)
        if (!handled && !is.null(test)) {
            skip_empty()
        }
    }, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, 
        message = handle_message, error = handle_error), error = handle_fatal, 
        skip = function(e) {
        })
13: test_code(NULL, exprs, env)
12: source_file(path, child_env(env), wrap = wrap)
11: FUN(X[[i]], ...)
10: lapply(test_paths, test_one_file, env = env, wrap = wrap)
9: doTryCatch(return(expr), name, parentenv, handler)
8: tryCatchOne(expr, names, parentenv, handlers[[1L]])
7: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatch(code, testthat_abort_reporter = function(cnd) {
       cat(conditionMessage(cnd), "\n")
       NULL
   })
5: with_reporter(reporters$multi, lapply(test_paths, test_one_file, 
       env = env, wrap = wrap))
4: test_files_serial(test_dir = test_dir, test_package = test_package, 
       test_paths = test_paths, load_helpers = load_helpers, reporter = reporter, 
       env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, 
       wrap = wrap, load_package = load_package)
3: test_files(test_dir = path, test_paths = test_paths, test_package = package, 
       reporter = reporter, load_helpers = load_helpers, env = env, 
       stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, 
       wrap = wrap, load_package = load_package, parallel = parallel)
2: test_dir("testthat", package = package, reporter = reporter, 
       ..., load_package = "installed")
1: test_check("irace", reporter = c("summary", "check"))
== irace == An unexpected condition occurred. Please report this bug to the authors of the irace package <https://github.com/MLopez-Ibanez/irace/issues>
Backtrace:
    ▆
 1. └─irace (local) cap.irace(...) at test-capping.R:76:2
 2.   └─irace::irace(scenario = scenario, parameters = parameters) at test-capping.R:57:2
 3.     └─irace:::irace_common(scenario, parameters, simple = TRUE) at irace/R/irace.R:682:9
 4.       └─irace:::irace_run(scenario = scenario, parameters = parameters) at irace/R/irace.R:710:2
 5.         └─irace:::elitist_race(...) at irace/R/irace.R:1264:4
 6.           └─irace:::irace.assert(all(colAnys(!is.na(elite.data)))) at irace/R/race.R:719:4
 7.             └─irace:::irace.internal.error(msg) at irace/R/utils.R:104:2
MLopez-Ibanez commented 1 year ago

I have not seen this issue for a while so it has probably been fixed by other changes.