`check_optimization_results()` sometimes crashes CBC

gewesp commented 2 years ago

Describe the bug

check_optimization_result()crashes CBC on some models. From the error message, it looks like an off-by-one error.

I'm calling the function after verifying that an optimal solution was found:

    if prob.model.num_solutions < 1 or mip.OptimizationStatus.OPTIMAL != status:
        raise RuntimeError(f"No solution found, status: {status}")

    prob.model.check_optimization_results()

Apart from that, my solutions look perfectly fine; although I noticed today that occasionally the progress log is nondeterministic---Are any randomized algorithms involved?

Edit: Repro steps added. I had some constraints in my model that evaluated to something like 0 <= 1. Cbc probably decides to discard those, but Python-MIP doesn't notice.

Workaround: Don't add tautological constraints like in the repro steps below.

Error message and stack trace:

Invalid row index (4086), valid range is [0,4086). At [...]/cbc/cbc/Cbc/src/Cbc_C_Interface.cpp:2410

ERROR while running Cbc. Signal SIGABRT caught. Getting stack trace.
0   libCbc.0.dylib                      0x000000011d3419ac _Z15CbcCrashHandleri + 296
1   libsystem_platform.dylib            0x00000001a5e6c4e4 _sigtramp + 56
2   libsystem_pthread.dylib             0x00000001a5e54eb0 pthread_kill + 288
3   libsystem_c.dylib                   0x00000001a5d92314 abort + 164
4   libCbc.0.dylib                      0x000000011d3fac84 Cbc_getRowIndices.cold.1 + 0
5   libCbc.0.dylib                      0x000000011d3a9b0c Cbc_getRowIndices + 0
6   libffi.dylib                        0x00000001b3ed8050 ffi_call_SYSV + 80
7   libffi.dylib                        0x00000001b3ee09e4 ffi_call_int + 948
8   _cffi_backend.cpython-39-darwin.so  0x00000001072659fc cdata_call + 1092
9   Python                              0x000000010589cd64 _PyObject_MakeTpCall + 360
10  Python                              0x0000000105972e00 call_function + 512
11  Python                              0x0000000105970418 _PyEval_EvalFrameDefault + 23080
12  Python                              0x000000010589d55c function_code_fastcall + 112
13  Python                              0x0000000105972da0 call_function + 416
14  Python                              0x00000001059703f4 _PyEval_EvalFrameDefault + 23044
15  Python                              0x000000010589d55c function_code_fastcall + 112
16  Python                              0x00000001058a68a8 property_descr_get + 128
17  Python                              0x00000001058e3d8c _PyObject_GenericGetAttrWithDict + 196
18  Python                              0x000000010596eb08 _PyEval_EvalFrameDefault + 16664
19  Python                              0x000000010589d55c function_code_fastcall + 112
20  Python                              0x0000000105972da0 call_function + 416
21  Python                              0x00000001059703f4 _PyEval_EvalFrameDefault + 23044
22  Python                              0x000000010589d55c function_code_fastcall + 112
23  Python                              0x0000000105972da0 call_function + 416
24  Python                              0x0000000105970418 _PyEval_EvalFrameDefault + 23080
25  Python                              0x000000010589d55c function_code_fastcall + 112
26  Python                              0x0000000105972da0 call_function + 416
27  Python                              0x0000000105970418 _PyEval_EvalFrameDefault + 23080
28  Python                              0x000000010589d55c function_code_fastcall + 112
29  Python                              0x0000000105972da0 call_function + 416
30  Python                              0x0000000105970494 _PyEval_EvalFrameDefault + 23204
31  Python                              0x000000010589d55c function_code_fastcall + 112
32  Python                              0x0000000105972da0 call_function + 416
33  Python                              0x0000000105970494 _PyEval_EvalFrameDefault + 23204
34  Python                              0x0000000105973cc8 _PyEval_EvalCode + 2988
35  Python                              0x000000010596a928 PyEval_EvalCode + 80
36  Python                              0x00000001059b27b4 pyrun_file + 304
37  Python                              0x00000001059b0788 PyRun_SimpleFileExFlags + 624
38  Python                              0x00000001059cda7c Py_RunMain + 1700
39  Python                              0x00000001059cdf40 pymain_main + 340
40  Python                              0x00000001059cdfbc Py_BytesMain + 40
41  dyld                                0x00000001050610f4 start + 520

To Reproduce

Minimal example:

import mip
model = mip.Model()

model.add_constr(mip.xsum([]) <= 1)

for c in model.constrs:
    print(c)

Expected behavior

Function doesn't crash CBC.

Laptop:

Operating System, version: macOS Monterey Version 12.1 (21C52)
Python version: 3.9.9
Python-MIP version (we recommend you to test with the latest version): 1.13.0
CBC version: Built with ../coinbrew --ssh -j8 build Cbc@master on Mar 15, 2022:
```
Version: devel 
Build Date: Mar 15 2022
```

gewesp commented 2 years ago

Update: I can narrow it down to iterating over model.constrs. It happens also before calling optimize(). It appears that Python-MIPs idea of the number of constraints does not agree with Cbc's. Something fishy with cbclib.Cbc_getNumRows().

Interestingly, I now get an Abort without a stack trace and in a different location in the C++ code:

BOPOPT:INFO: Model has 220 constraint(s)
BOPOPT:INFO: Solver has 219 constraint(s)
Invalid row index (219), valid range is [0,219). At .../cbc/cbc/Cbc/src/Cbc_C_Interface.cpp:1599
Abort trap: 6

With the following code:

    assert n_constr == model.num_rows
    logger.info(f"Model has {n_constr} constraint(s)")
    logger.info(f"Solver has {model.solver.num_rows()} constraint(s)")
    logger.debug("All constraints:")
    for c in model.constrs:
        logger.debug(f"{c}")

Before that, I call nothing special, just calls to add_var() and add_constr().

sebheger commented 2 years ago

@gewesp Thanks for your investigations. We are already aware of the problem that handling of so-called "empty" constraints at python-mip as well at CBC interface is not correct. See https://github.com/coin-or/python-mip/pull/237 where I proposed a fix on python-mip side, that is still "pending" as handling of this cases by the cbc api is not 100% clear.

coin-or / python-mip

`check_optimization_results()` sometimes crashes CBC #254