guillermo-navas-palencia
commented
8 months ago One of the fastest greedy (local-search / heuristic) solvers available in the market is LocalSolver (commercial). This solver is supported, but you need a licence. See this example: http://gnpalencia.org/optbinning/tutorials/tutorial_binary_localsolver.html.
Suppose the number of prebins is large (say 100). In that case, the problem has many decision variables, and it is unlikely you can solve it in the order of milliseconds, especially if there are several constraints involved. Here are a few options to consider:
- Are the 100 prebins necessary to characterize a good initial solution? I would assume you might be using quantiles or evenly-spaced bins. Did you try using 'cart' as a prebinning method? Are the results comparable?
- If your problem does not require many constraints, consider devising a heuristic such as a custom decision tree.
- Set time_limit <= 1 second (the solver might need more time to find a feasible solution).
- Use the BinningProcess class with multiple jobs. http://gnpalencia.org/optbinning/tutorials/tutorial_binary_localsolver.html
Developing reliable heuristics satisfying constraints does require a lot of expertise. The underlying problem can be seen as a generalized assignment problem, and you can always try to relax the integrality constraints and try fixing variables. I think this type of problem is better handled using solvers.
Are there any plans to include an option for a (sub-optimal) fast, greedy solver? I often have ~1000 fields in exploratory analysis which is expensive at 5-10s per field with the 'cp' solver. Typically I'm optimizing from about 100 to 10 bins.