Pyomo / pyomo

An object-oriented algebraic modeling language in Python for structured optimization problems.
https://www.pyomo.org
Other
2.01k stars 517 forks source link

Thread handling in Pyomo #2989

Open ZedongPeng opened 1 year ago

ZedongPeng commented 1 year ago

Summary

I am doing some benchmark tests with MindtPy. For the fairness of these benchmarks, I want to limit each benchmark test using only one thread. However, I found a weird situation in Pyomo. I can successfully pass the threads option to solvers like CPLEX and solvers will only use one thread. However, at the stage before CPLEX solves the model. Multiple threads are used. Is there some magic stuff like generating LP inside Pyomo using multiple threads?

Steps to reproduce the issue

  1. Download the unitcommit_200_100_2_mod_7 example.
  2. Use the following Python script to solve it.
    
    # example.py
    from pyomo.environ import *
    from importlib import import_module

module = import_module("unitcommit_200_100_2_mod_7") opt = SolverFactory('cplex') opt.options['threads'] = 1 results = opt.solve(module.m, tee=True) print(results)


If you run the above code, you will find that multiple threads are involved at the beginning. After that, 'CPLEX' uses only one thread to solve the model.

### Information on your system

Pyomo version: 6.6.2.dev0
Python version: 3.10.12
Operating system: macOS
How Pyomo was installed (PyPI, conda, source): source
Solver (if applicable): CPLEX

### Additional information

<!-- If you have any additional information, please list it here. -->

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question (how to use Pyomo, what it can do, etc.), please refer to our [online documentation](https://github.com/Pyomo/pyomo#getting-help) on ways to get help. -->
ZedongPeng commented 1 year ago

I checked again. Multi threads are used when building pyomo models. More exactly, this line. module = import_module("unitcommit_200_100_2_mod_7")

jsiirola commented 2 months ago

How are you identifying that multiple threads are being used? While Pyomo does make use of multithreading to manage the console output from subprocess solves (particularly for tee=True), there is nothing in the Pyomo code that should invoke multithreading while importing the model. Further, looking at top on Linux, I don't see the python process exceeding 100% CPU (which again doesn't point to multithreading).