Closed ccocheci closed 6 years ago
Since this is a global configuration in pyutilib, what about a function that disables signal management?
Do you have a simple example that illustrates this behavior? I'm not sure how to test this.
This functionality should already exist. run_command
has an optional argument define_signal_handlers
. It defaults to True
and setting False
should prevent the signal handlers from being set up.
Do we need a separate functionality outside of the run_command() logic? That's not obvious from the ticket description?
I observed this problem while trying to execute a pyomo-based function in a thread, I can provide a simple example.
Here's a simple optimization script that uses pyomo (example_opt.py):
from pyomo.environ import *
def opt(output_path):
model = ConcreteModel()
model.x = Var()
model.constraint = Constraint(expr=0 <= model.x <= 10)
model.objective = Objective(expr=model.x)
# this is where the error "signal only works in main thread" happens
result = SolverFactory("glpk").solve(model)
xvalue = value(model.x)
output_path1 = output_path
return xvalue,output_path1
if __name__ == '__main__':
output_path = 'folder'
x,path = opt(output_path)
print('x = ' +str(x))
print('path = ' + str(path))
And here's the script that invokes it in a Thread:
import os
import example_opt
from threading import Thread
def calc_test(output_path):
thread = Thread(target=example_opt.opt,args = (output_path,))
thread.start()
thread.join()
if __name__ == '__main__':
output_path = 'new_Outputs'
if not os.path.exists(output_path):
os.makedirs(output_path,exist_ok=True)
calc_test(output_path)
OK, this is a fun one. The problem is with Pyomo - it is asking for signal hangling - but the easiest place to fix this is in PyUtilib. We don't want to turn off signal handling - when we do, killing Pyomo with a signal (usually Ctrl-C) leaves the solver subprocess running. I see ~4 options:
We could go into every place that Pyomo calls pyutilib.subprocess.processmngr.run_command
and add try-except traps to automatically disable signal handling - but that is going to be hard to ensure that both we get all of them, and that no new calls sneak in. It will also be hard to trap: the signals system apparently raises a ValueError
in this case. That is a rather common exception, and trapping that to retry without signals
We could add a global USE_SUBPROCESS_SIGNAL_HANDLERS
flag to Pyomo that defaults to True, but users like @ccocheci could set to False. The downside is that we would need to pass that global flag to every call to run_command
(with all the same problems as above)
We could add a try-except block into run_command
itself so that it automatically falls back on disabling signal handling when it cannot set up the signals. I was going to propose this, but the more I think about it, the less I like it: bad things can happen if signals aren't propagated to the subprocess - including leaving solvers running after timelimits. This would be especially evil for folks running something like Pyomo in a server environment. think that the user (well, downstream developer) should have to take positive action to disable the signal handler.
Add a global SUBPROCESS_USE_SIGNAL_HANDLERS_DEFAULT
(or any other semantically equivalent name) to Pyutilib. This would change the default value for the run_command. I think I like this the best:
a. It requires positive action by the downstream developer to turn off signal handling
b. It is a single central fix, so we don't have to go update everywhere run_command is called
@whart222: What do you think of solution 4? That should be a minimally-invasive change that should make everyone happy.
@whart222 Is this change available in Pyomo also now? Because still I am getting 'signal only works in main thread' error in Pyomo 5.5.0.
Yes, as long as you have PyUtilib 5.6.3, you have this fix. That said, signal handlers are still on by default. If you want to turn it off, you need to:
import pyutilib.subprocess.GlobalData
pyutilib.subprocess.GlobalData.DEFINE_SIGNAL_HANDLERS_DEFAULT = False
Somewhere in your script before invoking a solver.
Even if this discussion is already a bit older, I'd like to share how circumvented the signal handling problem.
The "problematic code" (here result = SolverFactory("glpk").solve(model)
) can be executed in a separate process that is created on the fly and is able to communicate to the main process using signals.
Do something like this
with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor:
return_value = executor.submit(SolverFactory("glpk").solve, model)
result = return_value.result()
Because pyutilib handles signals by default, it cannot run on a child thread. This is very restricting when trying to run anything based on pyutilib in server-side code, which usually instantiates new threads to allow the serving of multiple clients. It would be very helpful to be able to pass a parameter to pyutilib that disables signal handling.