Open kdd0211 opened 6 years ago
@kdd0211 Strange, I don't have any object called init
in this package.
Do you know whether you defined the bounds of the inputs? (in the case the loss function does not handle very large values, just to check)
The best we could do, if you are able to, is to use in RStudio when your script got the error interactively:
As it is a long computation I wonder whether you are able to rerun it (522 * 10 seconds => nearly 2 hours... x2 if Rerun with Debug in RStudio).
To do the debugging in RStudio, try to familiriaze yourself using this toy data to run from a script (paste the lines below in a script, select all lines, then press Ctrl+Enter => in the Console, you have Show Traceback, and Rerun with Debug):
abc <- function() {
plot(NA)
}
abc()
In the debugger (if you never used it), you have the traceback on the right to check all the existing environments the error is in.
Use recover
to change environments. Example:
> abc()
Error in plot.window(...) : need finite 'ylim' values
In addition: Warning messages:
1: In min(x) : no non-missing arguments to min; returning Inf
2: In max(x) : no non-missing arguments to max; returning -Inf
Called from: plot.window(...)
Browse[1]> recover()
Enter a frame number, or 0 to exit
1: abc()
2: #2: plot(NA)
3: plot.default(NA)
4: localWindow(xlim, ylim, log, asp, ...)
5: plot.window(...)
6: (function ()
{
.rs.breakOnError(TRUE)
})()
7: .rs.breakOnError(TRUE)
8: eval(substitute(browser(skipCalls = pos), list(pos = (length(sys.frames()) - frame) + 2)), envir = sys.frame(frame))
9: eval(substitute(browser(skipCalls = pos), list(pos = (length(sys.frames()) - frame) + 2)), envir = sys.frame(frame))
Selection:
Thanks for the quick reply!
[1] What do you mean by the bounds of the inputs? I don’t think I defined those. Below is my invocation of the function (code is attached).
result <- CEoptim(MMLCodefn_disc, continuous = list(mean=rep(5, condim), sd=rep(1.5, condim)), discrete = list(probs = p0), rho = 0.1, N = 200L, verbose = TRUE,parallelize = TRUE,cl = cl,noImproveThr=5)
Earlier in the code, I define p0, the probabilities associated with the discrete sampling
p0 <- list() for (i in 1:length(testmu)) { p0 <- c(p0, list(c(0.7, 0.3))) }
[2] I just ran traceback() in the console and this is what happened …
traceback() 4: eval(ei, envir) 3: eval(ei, envir) 2: withVisible(eval(ei, envir)) 1: source("C:/Users/kevin/Google Drive/McFarlane/McMikeOptimization_single_stochastic_par02.R")
Do you have any idea what is going on?
[3] Are you suggesting I rerun McMikeOptimization_single_stochastic_par02.R in debug mode? I am happy to do that if so …
Sent from Mail for Windows 10
From: Laurae Sent: Wednesday, March 21, 2018 1:23 PM To: Laurae2/LauraeCE Cc: kdd0211; Mention Subject: Re: [Laurae2/LauraeCE] Mixed optimization error (#1)
@kdd0211 Strange, I don't have any object called init in this package. Do you know whether you defined the bounds of the inputs? (in the case the loss function does not handle very large values, just to check) The best we could do, if you are able to, is to use in RStudio when your script got the error interactively: • Show Traceback: do you find where it interrupted? If yes, could you paste all the traceback here? • Rerun with Debug: this would allow you to debug interactively at the point it encountered the error As it is a long computation I wonder whether you are able to rerun it (522 * 10 seconds => nearly 2 hours... x2 if Rerun with Debug in RStudio). To do the debugging in RStudio, try to familiriaze yourself using this toy data to run from a script (paste the lines below in a script, select all lines, then press Ctrl+Enter => in the Console, you have Show Traceback, and Rerun with Debug): abc <- function() { plot(NA) }
abc() In the debugger (if you never used it), you have the traceback on the right to check all the existing environments the error is in.
Use recover to change environments. Example:
abc() Error in plot.window(...) : need finite 'ylim' values In addition: Warning messages: 1: In min(x) : no non-missing arguments to min; returning Inf 2: In max(x) : no non-missing arguments to max; returning -Inf Called from: plot.window(...) Browse[1]> recover()
Enter a frame number, or 0 to exit
1: abc() 2: #2: plot(NA) 3: plot.default(NA) 4: localWindow(xlim, ylim, log, asp, ...) 5: plot.window(...) 6: (function () { .rs.breakOnError(TRUE) })() 7: .rs.breakOnError(TRUE) 8: eval(substitute(browser(skipCalls = pos), list(pos = (length(sys.frames()) - frame) + 2)), envir = sys.frame(frame)) 9: eval(substitute(browser(skipCalls = pos), list(pos = (length(sys.frames()) - frame) + 2)), envir = sys.frame(frame))
Selection: — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
The error comes from source
function, because it calls eval
, with exactly the parameters the error message you provided.
Do you have any idea what is going on?
Run the code without source
. It must print out significantly much more details for troubleshooting the issue. Otherwise, the code is wrapped in an eval
and we can't attempt to guess a solution.
Are you suggesting I rerun McMikeOptimization_single_stochastic_par02.R in debug mode? I am happy to do that if so …
Ideally yes, preferable without sourcing it (just paste the code in a console to get access to detailed information - paste until CEoptim
code, then run manually your CEoptim
line code)
Interestingly, it actually finished without any error when I run without sourcing. Let me try again.
The very first time I ran this program (via sourcing) it also worked but most of the time (I’d say around 90%) it does not.
… Wed Mar 21 2018 6:48:13 PM - iter: 00516 (11s045ms, 18.11 samples/s, 2.59 s/s/thread) - opt: -2663.561 - maxSd: 0.002078089 - maxProbs: 0 Wed Mar 21 2018 6:48:25 PM - iter: 00517 (11s842ms, 16.89 samples/s, 2.41 s/s/thread) - opt: -2663.561 - maxSd: 0.001772977 - maxProbs: 0 Wed Mar 21 2018 6:48:36 PM - iter: 00518 (11s592ms, 17.25 samples/s, 2.46 s/s/thread) - opt: -2663.561 - maxSd: 0.001220485 - maxProbs: 0 Wed Mar 21 2018 6:48:47 PM - iter: 00519 (11s104ms, 18.01 samples/s, 2.57 s/s/thread) - opt: -2663.561 - maxSd: 0.001112843 - maxProbs: 0 Wed Mar 21 2018 6:48:59 PM - iter: 00520 (11s523ms, 17.36 samples/s, 2.48 s/s/thread) - opt: -2663.561 - maxSd: 0.001120907 - maxProbs: 0 Wed Mar 21 2018 6:49:10 PM - iter: 00521 (10s981ms, 18.21 samples/s, 2.60 s/s/thread) - opt: -2663.561 - maxSd: 0.001119486 - maxProbs: 0 Wed Mar 21 2018 6:49:21 PM - iter: 00522 (11s586ms, 17.26 samples/s, 2.47 s/s/thread) - opt: -2663.561 - maxSd: 0.0009323155 - maxProbs: 0
}
stopCluster(cl)
closeAllConnections()
final=Sys.time()
time_elapsed=final-init
print(time_elapsed) Time difference of 1.602298 hours
Will rerun again (without sourcing) and let you know.
Sent from Mail for Windows 10
From: Laurae Sent: Wednesday, March 21, 2018 5:04 PM To: Laurae2/LauraeCE Cc: kdd0211; Mention Subject: Re: [Laurae2/LauraeCE] Mixed optimization error (#1)
The error comes from source function, because it calls eval, with exactly the parameters the error message you provided. Do you have any idea what is going on? Run the code without source. It must print out significantly much more details for troubleshooting the issue. Otherwise, the code is wrapped in an eval and we can't attempt to guess a solution. Are you suggesting I rerun McMikeOptimization_single_stochastic_par02.R in debug mode? I am happy to do that if so … Ideally yes, preferable without sourcing it (just paste the code in a console to get access to detailed information - paste until CEoptim code, then run manually your CEoptim line code) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
It finished again, but now when I run it tweaking the discrete probabilities, it crashes (running without sourcing). I get the following error.
Wed Mar 21 2018 9:37:59 PM - iter: 00001 (14s142ms, 14.14 samples/s, 2.02 s/s/thread) - opt: -1735.637 - maxSd: 2.153206 - maxProbs: 0.3 Error in if (Y[elite[nElite]] < gammat) { : missing value where TRUE/FALSE needed
These are the pertinent lines of code. I tweaked the discrete probabilities from 0.7/0.3 to 0.9/0.1. (Commented out line is what it was previously)
p0 <- list() for (i in 1:length(testmu)) { p0 <- c(p0, list(c(0.9, 0.1)))
}
And below is the invocation of CEoptim:
result <- CEoptim(MMLCodefn_disc, continuous = list(mean=rep(5, condim), sd=rep(1.5, condim)), discrete = list(probs = p0), rho = 0.1, N = 200L, verbose = TRUE,parallelize = TRUE,cl = cl,noImproveThr=5)
My understanding is that this is telling CEoptim that an arbitrary element of the discrete vector we are optimizing over is equal to 1 with probability 0.9, and 0 with probability 0.1. Is that correct? Why does it work instead when these probabilities are 0.7 and 0.3 respectively?
Sent from Mail for Windows 10
From: Kevin D. Dayaratna Sent: Wednesday, March 21, 2018 7:49 PM To: Laurae2/LauraeCE Subject: RE: [Laurae2/LauraeCE] Mixed optimization error (#1)
Interestingly, it actually finished without any error when I run without sourcing. Let me try again.
The very first time I ran this program (via sourcing) it also worked but most of the time (I’d say around 90%) it does not.
… Wed Mar 21 2018 6:48:13 PM - iter: 00516 (11s045ms, 18.11 samples/s, 2.59 s/s/thread) - opt: -2663.561 - maxSd: 0.002078089 - maxProbs: 0 Wed Mar 21 2018 6:48:25 PM - iter: 00517 (11s842ms, 16.89 samples/s, 2.41 s/s/thread) - opt: -2663.561 - maxSd: 0.001772977 - maxProbs: 0 Wed Mar 21 2018 6:48:36 PM - iter: 00518 (11s592ms, 17.25 samples/s, 2.46 s/s/thread) - opt: -2663.561 - maxSd: 0.001220485 - maxProbs: 0 Wed Mar 21 2018 6:48:47 PM - iter: 00519 (11s104ms, 18.01 samples/s, 2.57 s/s/thread) - opt: -2663.561 - maxSd: 0.001112843 - maxProbs: 0 Wed Mar 21 2018 6:48:59 PM - iter: 00520 (11s523ms, 17.36 samples/s, 2.48 s/s/thread) - opt: -2663.561 - maxSd: 0.001120907 - maxProbs: 0 Wed Mar 21 2018 6:49:10 PM - iter: 00521 (10s981ms, 18.21 samples/s, 2.60 s/s/thread) - opt: -2663.561 - maxSd: 0.001119486 - maxProbs: 0 Wed Mar 21 2018 6:49:21 PM - iter: 00522 (11s586ms, 17.26 samples/s, 2.47 s/s/thread) - opt: -2663.561 - maxSd: 0.0009323155 - maxProbs: 0
}
stopCluster(cl)
closeAllConnections()
final=Sys.time()
time_elapsed=final-init
print(time_elapsed) Time difference of 1.602298 hours
Will rerun again (without sourcing) and let you know.
Sent from Mail for Windows 10
From: Laurae Sent: Wednesday, March 21, 2018 5:04 PM To: Laurae2/LauraeCE Cc: kdd0211; Mention Subject: Re: [Laurae2/LauraeCE] Mixed optimization error (#1)
The error comes from source function, because it calls eval, with exactly the parameters the error message you provided. Do you have any idea what is going on? Run the code without source. It must print out significantly much more details for troubleshooting the issue. Otherwise, the code is wrapped in an eval and we can't attempt to guess a solution. Are you suggesting I rerun McMikeOptimization_single_stochastic_par02.R in debug mode? I am happy to do that if so … Ideally yes, preferable without sourcing it (just paste the code in a console to get access to detailed information - paste until CEoptim code, then run manually your CEoptim line code) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
My understanding is that this is telling CEoptim that an arbitrary element of the discrete vector we are optimizing over is equal to 1 with probability 0.9, and 0 with probability 0.1. Is that correct?
It's exact.
Why does it work instead when these probabilities are 0.7 and 0.3 respectively?
No idea.
Error in if (Y[elite[nElite]] < gammat) { : missing value where TRUE/FALSE needed
You are returning a NA or NaN value in your loss function. Is your function "shielded" against non sense returns?
Edge cases you need to take into account for your loss function:
You can add the following before the return of your loss function:
if (is.finite(my_return) {
return(my_return)
} else {
browser()
return(input any absurdly large value here)
}
If you happen to end up in the debugger while running, it means the loss function attempted to return a non-finite value (NA, NaN, Inf). Run your loss function manually for the edge cases and check whether you are getting an error (without running CEoptim, there must be an unhandled case in your loss function).
Greetings. I keep encountering this error each time I run a mixed integer (continuous plus discrete) parallelized optimization using CEOptim. Do you have any idea what this is and how I can fix it?
Tue Mar 20 2018 9:27:49 PM - iter: 00521 (10s625ms, 18.82 samples/s, 2.69 s/s/thread) - opt: -2663.561 - maxSd: 0.001119486 - maxProbs: 0 Tue Mar 20 2018 9:27:59 PM - iter: 00522 (10s740ms, 18.62 samples/s, 2.66 s/s/thread) - opt: -2663.561 - maxSd: 0.0009323155 - maxProbs: 0 Error in eval(ei, envir) : object 'init' not found