search

coin-or / Clp

COIN-OR Linear Programming Solver
Other
412 stars 84 forks source link

1.17.5 exception in coin LP: Bad new rownum #130

Open Mizux opened 4 years ago

Mizux commented 4 years ago

When bumping google/ortools dependencies from 1.17.3 to 1.17.5 https://github.com/google/or-tools/commit/a5caa9a7308a0aa4ef6c87dfdca1752e446fa10c (ed this is using the autotools based build to generate the coin clp library not my CMake based build fork)

We have a dumb LP test https://github.com/google/or-tools/blob/858fa626959f7e386153af82756384b79f983b5a/examples/tests/lp_test.cc#L53-L83

which now fails with the trace:

%make run SOURCE=examples/tests/lp_test.cc 
bin/lp_test 
I0206 14:16:57.725375 76306 lp_test.cc:137] ---- Linear programming example with CLP ----
I0206 14:16:57.725632 76306 lp_test.cc:25] Number of variables = 2
I0206 14:16:57.725641 76306 lp_test.cc:26] Number of constraints = 3
W0206 14:16:57.725795 76306 clp_interface.cc:508] Caught exception in Coin LP: Bad new rownum (less than current)
F0206 14:16:57.725806 76306 lp_test.cc:31] The problem does not have an optimal solution!
*** Check failure stack trace: ***
    @     0x7f31bccaf21c  google::LogMessage::Fail()
    @     0x7f31bccaf17a  google::LogMessage::SendToLog()
    @     0x7f31bccaead9  google::LogMessage::Flush()
    @     0x7f31bccb1f56  google::LogMessageFatal::~LogMessageFatal()
    @     0x563ab7653e40  operations_research::SolveAndPrint()
    @     0x563ab7654249  operations_research::RunLinearProgrammingExample()
    @     0x563ab7654979  operations_research::RunAllExamples()
    @     0x563ab765386b  main
    @     0x7f31bbc62bbb  __libc_start_main
    @     0x563ab76538aa  _start
    @              (nil)  (unknown)
make: *** [makefiles/Makefile.cpp.mk:282: run] Aborted

Seems to pass using CLP tag 1.17.4, so I "think" there is a regression between both tags... Currently investigating....

note: could be related to #113 IMHO

svigerske commented 4 years ago

Yes, the only difference between these releases is d167538f

Mizux commented 4 years ago

Does it means that our wrapper is wrong ? i.e. https://github.com/google/or-tools/blob/858fa626959f7e386153af82756384b79f983b5a/ortools/linear_solver/clp_interface.cc#L404-L512

svigerske commented 4 years ago

I don't know. I only wanted to point you directly to the change in Clp. It's not too unlikely that this fix (for #113) has broken something. If you can generate a standalone driver to reproduce this problem, that could help.

Mizux commented 4 years ago

after investigation this error is throw in this call:

clp_->addRows(build_object); ref: https://github.com/google/or-tools/blob/45770b833997f827d322e929b1ed4781c4e60d44/ortools/linear_solver/clp_interface.cc#L380

note: before this call if adding a clp_->matrix() I get a null pointer i.e.there is no packed matrix at this point...

Mizux commented 4 years ago

First "bug": we have https://github.com/google/or-tools/blob/45770b833997f827d322e929b1ed4781c4e60d44/ortools/linear_solver/clp_interface.cc#L380 calling https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpModel.cpp#L1915 which contains https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpModel.cpp#L1968-L1972 line:1968 this will call syncronizeMatrix() internally then line:1971-1972 may increase the matrix i.e. matrix.rows() > numberRows_ member then the exception occurs there l:2086 since you want to "reduce" the matrix size which is forbidden https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpModel.cpp#L2086-L2088 further more you have a call to synchronizeMatrix() just after...

Now I have a throw when calling initialSolve(option), still investigating...

Mizux commented 4 years ago

It throw again using this code path: https://github.com/google/or-tools/blob/45770b833997f827d322e929b1ed4781c4e60d44/ortools/linear_solver/clp_interface.cc#L453 calling https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpSolve.cpp#L848 calling https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpSolve.cpp#L975-L976 calling https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpPresolve.cpp#L103-L110 calling https://github.com/coin-or/Clp/blob/756ddd3ed813eb1fa8b2d1b4fe813e6a4d7aa1eb/Clp/src/ClpPresolve.cpp#L114-L115

psarka commented 3 years ago

Probably won't be of much help, but if you need a failing piece of code then this https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp fails with the same exception when using coin-or from conan https://conan.io/center/coin-clp.

It does not, however, fail when using binary from https://www.coin-or.org/download/binary/Clp/Clp-1.17.6-linux-x86_64-gcc5.4.0.tgz which I can't comprehend.

Edit: ok, one puzzle solved - I actually did not try the 1.17.6 binary from the coin-or.org, rather in a quite a confusing manner I included the coin-or that was distributed by the ortools. So 1.17.6 binary might be broken addRow-wise as well.

Mizux commented 3 years ago

@psarka IIRC master branch is a complete rewrite of the stable/1.17 branch so it may explain few things.

Otherwise we may need a diff between 1.17.5 and 1.17.6 https://github.com/coin-or/Clp/compare/releases/1.17.5...releases/1.17.6

tkralphs commented 3 years ago

@jjhforrest Can you take a look at this? It would be nice to get it cleared up.

jjhforrest commented 3 years ago

Unable to reproduce exact bug in 1.17.5.

I have found some problems when adding rows/columns in odd ways. I have updated ClpModel.cpp in stable/1.17. It is safe to copy ClpModel.cpp to code in 1.17.5 or 1.17.6

John Forrest On 28/09/2021 20:50, Mizux wrote:

@psarka https://github.com/psarka IIRC master branch is a complete rewrite of the stable/1.17 branch so it may explain few things.

Otherwise we may need a diff between 1.17.5 and 1.17.6

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/coin-or/Clp/issues/130#issuecomment-929573335, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWJYHAXUPBSG2UVKHD6CFTUEIL7TANCNFSM4KQ42MTA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Mizux commented 3 years ago

@jjhforrest did you manage to run https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp ? (like pointed by psarka) And what do you mean by "when adding rows/columns in odd ways" ?

Would you mind to take a look at our 11years old Clp wrapper ? does this code seems odd to you ? https://github.com/google/or-tools/blob/stable/ortools/linear_solver/clp_interface.cc#L282-L388

Did you test using tags release/1.17.4, release/1.17.5, release/1.17.6, and branch stable/1.17 ?

jjhforrest commented 3 years ago

Ran addRows.cpp with 1.17.5 (and stable/1/17) - error occurred - new ClpModel.cpp fixed.

On 30/09/2021 14:07, Mizux wrote:

@jjhforrest https://github.com/jjhforrest did you manage to run https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp https://github.com/coin-or/Clp/blob/master/examples/addRows.cpp ? like pointed by psarka ?

did you test release/1.17.4, release/1.17.5, release/1.17.6, stable/1.17 ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coin-or/Clp/issues/130#issuecomment-931304968, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWJYHGOXRIZ3A5BWPOWQZLUEROJDANCNFSM4KQ42MTA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Mizux commented 3 years ago

I'll try to update my fork (today?) on this patch and see if or-tools tests pass. I'll keep you posted...

note: This is a revamp of my comment in https://github.com/Mizux/Clp/commit/64ef8dca305fcab3f64ac2a6cdecbfa93104863d#commitcomment-57282017 with adding content (mostly the test with patch 44758f3).

I'm doing some test basically testing or-tools build with an "up to date" stable/1.17 branch (note or-tools is currently using the release/1.17.4 as top + add few CMake build stuff) ref: https://github.com/google/or-tools/blob/26bbc8856cac047c3171dcb15a2af8b16eb85dc1/cmake/dependencies/CMakeLists.txt#L178-L179

It was spotted that the or-tools is facing was introduced by the commit https://github.com/Mizux/Clp/commit/2ec532187a0187561447d958d122246e3d1530ff So I did some test like reverting this commit but using the stable/1.17 branch (kind of 1.17.6 at this time)

note: Bastian (conda maintainer tell me there is a fix in 1.17.6 for an other issue so wanted to test it and see if the release/1.17.5 regression was fixed)

My tests so far: branch or-tools test (bad new Rownum issue)
release/1.17.4 pass
stable/1.17 before 44758f3 patch fail
stable/1.17 before 44758f3 patch + revert pass
stable/1.17 with 44758f3 patch fail (but different issue)

note: Currently my branch stable/1.17 contains the patch 44758f3 as I try to keep it up to date with the coin-or branch (i.e. perform upstream git rebase)

stable1.17 with 44758f3 patch and CMake build, here the trace (when building in debug)

(gdb) run
Starting program: /home/mizux/work/Clp/build/Clp/test/ClpTest 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
coinutils version: 2.11.0
osi version: 0.108.0
clp version: 1.17.0
Clp0006I 0  Obj 0 Primal inf 1.8843531 (2)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow is 0.559329
Clp0006I 0  Obj 0 Primal inf 5470.6104 (10002)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow using CoinBuild is 0.001441
Clp0006I 0  Obj 0 Primal inf 5470.6104 (10002)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow using CoinBuild+-1 is 0.003226
Clp0006I 0  Obj 0 Primal inf 9999.999 (10000)
Clp0006I 1  Obj 1
Clp0000I Optimal - objective value 1
Time for 10000 addRow using CoinModel+-1 is 0.007595
Clp0006I 0  Obj 0 Primal inf 9999.999 (10000)
Clp0006I 1  Obj 1
Clp0000I Optimal - objective value 1
Time for 10000 addRow using CoinModel load +-1 is 0.009734
Clp0006I 0  Obj 0 Primal inf 9999.999 (10000)
Clp0006I 1  Obj 0
Clp0000I Optimal - objective value 0
Time for 10000 addRow using CoinModel is 0.010057
Clp0006I 0  Obj 0 Primal inf 5470.6104 (10002)
Clp0006I 2  Obj 2
Clp0000I Optimal - objective value 2
Time for 10000 addRow using hand written code is 0.000185
Clp0006I 0  Obj 0 Primal inf 1999.999 (10000)
Clp0006I 1  Obj 1
Clp0000I Optimal - objective value 1
               Primal          Dual         Lower         Upper          Cost
     0       2.000000      0.000000      0.000000      2.000000      1.000000
     1       0.200000      0.000000      0.0000000x1.fffffffffffffp+1023       0x0.p+0
     2        0x0.p+00x1.7ffffffffffffp+1       0x0.p+0       0x1.p+2       0x1.p+2
--------------------------------------
If Clp compiled without NDEBUG below should give assert, if with NDEBUG or COIN_ASSERT CoinError
ClpTest: /home/mizux/work/Clp/Clp/src/ClpModel.cpp:2028: int ClpModel::addRows(const CoinBuild&, bool, bool): Assertion `numberColumns <= numberColumns_' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff75a2d22 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff75a2d22 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff758c862 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff758c747 in __assert_fail_base.cold () from /usr/lib/libc.so.6
#3  0x00007ffff759b616 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff7e101df in ClpModel::addRows (this=0x7fffffffcb80, buildObject=..., tryPlusMinusOne=true, checkDuplicates=false) at /home/mizux/work/Clp/Clp/src/ClpModel.cpp:2028
#5  0x0000555555557d72 in main (argc=1, argv=0x7fffffffe1c8) at /home/mizux/work/Clp/Clp/test/ClpTest.cpp:277
(gdb)

todo(mizux): need to test with the project build system (i.e. CMake define/macro discrepancy issue ?) EDIT: tkralphs seems to has been able to reproduce it see below.

Protocol of the test

First I install mizux/CoinUtils and mizux/Osi in tmp/coin

git clone -b stable/2.11 git@github.com:Mizux/CoinUtils.git
cd CoinUtils
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/tmp/coin/usr/local
cmake --build build -v
cmake --build build --target install -v -- DESTDIR=/tmp/coin

git clone -b stable/0.108 git@github.com:Mizux/Osi.git
cd Osi
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/tmp/coin/usr/local
cmake --build build -v
cmake --build build --target install -v -- DESTDIR=/tmp/coin

Then I've built Clp using previously installed deps qnd run a test (ClpTest.cpp which is nearly the same than https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp) todo(mizux): Try to reduce the test sample

git clone -b stable/1.17 git@github.com:Mizux/Clp.git
cd Clp
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/tmp/coin/usr/local
cmake --build build -v
cmake --build build --target test
tkralphs commented 3 years ago

I just tried https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp with https://github.com/coin-or/Clp/commit/44758f3b33134001d1eeb42d8a177ca2a7dba1a0 (fixed stable/1.17) and it now fails for me with

Bad new colnum (less than current) in CoinPackedMatrix::setDimensions

@jjhforrest I assume you didn't see the same?

jjhforrest commented 3 years ago

Ted,

While debugging and modified, I did get that error with addRows.cpp.

Definitely did not get that error with modified ClpModel.cpp i.e. current Clp/stable/1.17

John On 03/10/2021 22:44, Ted Ralphs wrote:

I just tried https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp https://github.com/coin-or/Clp/blob/stable/1.17/Clp/examples/addRows.cpp with 44758f3 https://github.com/coin-or/Clp/commit/44758f3b33134001d1eeb42d8a177ca2a7dba1a0 (fixed stable/1.17) and it now fails for me with

|Bad new colnum (less than current) in CoinPackedMatrix::setDimensions |

@jjhforrest https://github.com/jjhforrest I assume you didn't see the same?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coin-or/Clp/issues/130#issuecomment-933030334, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWJYHBABXKAD4B7MLZ2TZ3UFDFCBANCNFSM4KQ42MTA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

tkralphs commented 3 years ago

Hmm. So I realized that the error being triggered was triggered intentionally and was meant to test the error mechanism itself (see comments in the code). I wasn't expecting such a thing in an example. So @jjhforrest, I guess you are seeing this later error but expecting it and so disregarding it?

So everything looks fine to me now. @Mizux, since you said your test is almost identical to addRows, so then I guess that your failure is also expected. If you look at the code, it is building a model with two columns and then trying to add rows with three columns, which is going to cause an error. It's not clear to me, though, why this would not happen with 1.17.4.

Edit: I get the same behavior with stable/1.17.4.

Mizux commented 3 years ago

  1. @tkralphs so if I understand you correctly, the current failing of addRows is expected so the current patch on stable/1.17 is OK. -> On you your side you seem ready to publish a release/1.17.7

  2. I've just do some test in or-tools (master branch) BUT targeting mizux/Clp stable/1.17 and we have issue when running sample (which was working until release/1.17.4) -> @tkralphs Should I deduce that our Clp wrapper is broken but luckily until 1.17.4 it was working ? -> todo(@mizux): need to fix our wrapper (my plan is to build a sample using only Clp but based on the or-tools wrapper code path then gently ask to the coin-or community what are we doing wrong then patch our wrapper accordingly for ortools v9.2 note: any better proposal welcome !!!

For the record sample: https://github.com/google/or-tools/blob/stable/examples/cpp/linear_programming.cc (which is simple and understandable) wrapper code: https://github.com/google/or-tools/blob/stable/ortools/linear_solver/clp_interface.cc (which is obscure for any non-Clp and non-or-tools developers) offtopic: maybe same wrong usage in cbc too https://github.com/google/or-tools/blob/stable/ortools/linear_solver/cbc_interface.cc

 %gdb build/bin/linear_programming
GNU gdb (GDB) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from build/bin/linear_programming...
(gdb) run
Starting program: /home/mizux/work/master/build/bin/linear_programming 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
I1006 15:15:57.887900  5289 linear_programming.cc:30] ---- Linear programming example with GLOP ----
I1006 15:15:57.893980  5289 linear_programming.cc:77] Number of variables = 3
I1006 15:15:57.894040  5289 linear_programming.cc:78] Number of constraints = 3
W1006 15:15:57.895765  5289 lp_solver.cc:163] 
******************************************************************
* WARNING: Glop will be very slow because it will use DCHECKs    *
* to verify the results and the precision of the solver.         *
* You can gain at least an order of magnitude speedup by         *
* compiling with optimizations enabled and by defining NDEBUG.   *
******************************************************************
I1006 15:15:57.900029  5289 linear_programming.cc:87] Problem solved in 7 milliseconds
I1006 15:15:57.900067  5289 linear_programming.cc:90] Optimal objective value = 733.333
I1006 15:15:57.900094  5289 linear_programming.cc:93] x1 = 33.3333
I1006 15:15:57.900107  5289 linear_programming.cc:94] x2 = 66.6667
I1006 15:15:57.900119  5289 linear_programming.cc:95] x3 = 0
I1006 15:15:57.900131  5289 linear_programming.cc:97] Advanced usage:
I1006 15:15:57.900141  5289 linear_programming.cc:98] Problem solved in 2 iterations
I1006 15:15:57.900153  5289 linear_programming.cc:99] x1: reduced cost = 0
I1006 15:15:57.900165  5289 linear_programming.cc:100] x2: reduced cost = 0
I1006 15:15:57.900176  5289 linear_programming.cc:101] x3: reduced cost = -2.66667
I1006 15:15:57.900204  5289 linear_programming.cc:103] c0: dual value = 3.33333 activity = 100
I1006 15:15:57.900219  5289 linear_programming.cc:105] c1: dual value = 0.666667 activity = 600
I1006 15:15:57.900232  5289 linear_programming.cc:107] c2: dual value = -0 activity = 200
I1006 15:15:57.900554  5289 linear_programming.cc:30] ---- Linear programming example with CLP ----
I1006 15:15:57.901302  5289 linear_programming.cc:77] Number of variables = 3
I1006 15:15:57.901327  5289 linear_programming.cc:78] Number of constraints = 3

Program received signal SIGSEGV, Segmentation fault.
ClpModel::addRows (this=0x55555558b7b0, buildObject=..., tryPlusMinusOne=false, checkDuplicates=true) at /home/mizux/work/master/build/_deps/clp-src/Clp/src/ClpModel.cpp:1971
1971          matrix_->setDimensions(-1, numberColumns_);
(gdb) bt full
#0  ClpModel::addRows (this=0x55555558b7b0, buildObject=..., tryPlusMinusOne=false, checkDuplicates=true) at /home/mizux/work/master/build/_deps/clp-src/Clp/src/ClpModel.cpp:1971
        numberElements = 9
        starts = 0x555555576000
        column = 0x555555583f00
        element = 0x55555557a200
        size = 0
        iRow = 3
        lower = 0x55555558b1d0
        upper = 0x555555571710
        __PRETTY_FUNCTION__ = "int ClpModel::addRows(const CoinBuild&, bool, bool)"
        number = 3
        numberErrors = 0
#1  0x00007ffff65e835e in operations_research::CLPInterface::ExtractNewConstraints (this=0x555555571b20) at /home/mizux/work/master/ortools/linear_solver/clp_interface.cc:379
        max_row_length = 3
        indices = std::unique_ptr<int []> = {get() = {<No data fields>}}
        coefs = std::unique_ptr<double []> = {get() = {<No data fields>}}
        build_object = {numberItems_ = 3, numberOther_ = 4, numberElements_ = 9, currentItem_ = 0x55555559abc0, firstItem_ = 0x55555559b2a0, lastItem_ = 0x55555559abc0, type_ = 0}
        total_num_rows = 3
#2  0x00007ffff661621c in operations_research::MPSolverInterface::ExtractModel (this=0x555555571b20) at /home/mizux/work/master/ortools/linear_solver/linear_solver.cc:1822
No locals.
#3  0x00007ffff65e8864 in operations_research::CLPInterface::Solve (this=0x555555571b20, param=...) at /home/mizux/work/master/ortools/linear_solver/clp_interface.cc:434
        timer = {running_ = true, start_ = 1633526157901340010, sum_ = 0}
        message_handler = {_vptr.CoinMessageHandler = 0x7ffff7ebeff0 <vtable for CoinMessageHandler+16>, doubleValue_ = std::vector of length 0, capacity 0, longValue_ = std::vector of length 0, capacity 0, 
          charValue_ = std::vector of length 0, capacity 0, stringValue_ = std::vector of length 0, capacity 0, logLevel_ = 0, logLevels_ = {-1000, 0, -1000, -1000}, prefix_ = 255, currentMessage_ = {externalNumber_ = -1, 
            detail_ = 0 '\000', severity_ = 73 'I', 
            message_ = "\000\000(ZWUUU", '\000' <repeats 11 times>, "ݙ\331\037\061\254\274P\327\377\377\377\177\000\000\235\340\374\365\377\177\000\000\000\000\000\000\000\000\000\000\220\327\377\377\377\177\000\000P\334\377\377\377\177\000\000\301\236\374\365\377\177\000\000\204\260ZUUU\000\000\215\241]a\000\000\000\000P\000\000\000\000U\000\000 ZWUUU\000\000\300\327\377\377\377\177\000\000\n\237`\366\377\177\000\000(ZWUUU\000\000\200\330\377\377\377\177\000\000\210\216YUUU\000\000\220\216YUUU\000\000\000\377\000\000\000\000\000\000\000ݙ\331\037\061\254\274\360\327\377\377\377\177\000\000\210\216YUUU\000\000p\216YUUU\000\000\000\330\377\377\377\177\000\000"...}, internalNumber_ = 0, format_ = 0x0, 
          messageBuffer_ = "\000PH\364\377\177\000\000\020\331\377\377\377\177\000\000o\316\353\366\377\177\000\000\350\214\372\367\377\177\000\000\240PH\364\377\177\000\000\210\216YUUU\000\000\377\377\000\000\016\000\000\000\223\323\300\a\204 :\360\001\000\000\000\016\000\000\000\016\000\000\000\001\000\000\000\000ݙ\331\037\061\254\274p\331\377\377\377\177\000\000Q\325\353\366\377\177\000\000\244\n\000\000\000\000\000\000\250PH\364\377\177\000\000P\331\377\377\377\177\000\000\031\375\353\366\377\177\000\000\244\n\000\000\016\000\000\000 PH\364\377\177\000\000\240\331\377\377\377\177\000\000o\316\353\366\377\177\000\000\350\214\372\367\377\177\000\000\240PH\364\377\177\000\000\000\000\000\000\003\000\000\000\000\000\000\000\016\000\000\000\223"..., messageOut_ = 0x7fffffffd8b8 "", source_ = "Unk", printStatus_ = 0, highestNumber_ = -1, fp_ = 0x7ffff40c4520 <_IO_2_1_stdout_>, g_format_ = "%.8g\000\177\000", g_precision_ = 8}
        tmp_status = 2
#4  0x00007ffff6612799 in operations_research::MPSolver::Solve (this=0x7fffffffdea0, param=...) at /home/mizux/work/master/ortools/linear_solver/linear_solver.cc:1406
        status = 3165401375
#5  0x00007ffff66126fc in operations_research::MPSolver::Solve (this=0x7fffffffdea0) at /home/mizux/work/master/ortools/linear_solver/linear_solver.cc:1392
        default_param = {static kDefaultDoubleParamValue = -1, static kDefaultIntegerParamValue = -1, static kUnknownDoubleParamValue = -2, static kUnknownIntegerParamValue = -2, static kDefaultRelativeMipGap = 0.0001, 
          static kDefaultPrimalTolerance = 9.9999999999999995e-08, static kDefaultDualTolerance = 9.9999999999999995e-08, static kDefaultPresolve = operations_research::MPSolverParameters::PRESOLVE_ON, 
          static kDefaultIncrementality = operations_research::MPSolverParameters::INCREMENTALITY_ON, relative_mip_gap_value_ = 0.0001, primal_tolerance_value_ = 9.9999999999999995e-08, dual_tolerance_value_ = 9.9999999999999995e-08, 
          presolve_value_ = 1, scaling_value_ = -1, lp_algorithm_value_ = -1, incrementality_value_ = 1, lp_algorithm_is_default_ = true}
#6  0x0000555555558c4f in operations_research::RunLinearProgrammingExample (solver_id="CLP") at /home/mizux/work/master/examples/cpp/linear_programming.cc:80
        problem_type = operations_research::MPSolver::CLP_LINEAR_PROGRAMMING
        solver = {_vptr.MPSolver = 0x7ffff7ea6bf0 <vtable for operations_research::MPSolver+16>, name_ = "IntegerProgrammingExample", problem_type_ = operations_research::MPSolver::CLP_LINEAR_PROGRAMMING, 
          interface_ = std::unique_ptr<operations_research::MPSolverInterface> = {get() = {<No data fields>}}, variables_ = std::vector of length 3, capacity 4 = {0x55555557a2d0, 0x5555555749d0, 0x5555555731c0}, 
          variable_name_to_index_ = std::optional<absl::lts_20210324::flat_hash_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, absl::lts_20210324::container_internal::StringHash, absl::lts_20210324::container_internal::StringHashEq::Eq, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >> [no contained value], 
          variable_is_extracted_ = std::vector<bool> of length 3, capacity 64 = {true, true, true}, constraints_ = std::vector of length 3, capacity 4 = {0x555555581a20, 0x55555557eda0, 0x555555581d60}, 
          constraint_name_to_index_ = std::optional<absl::lts_20210324::flat_hash_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, absl::lts_20210324::container_internal::StringHash, absl::lts_20210324::container_internal::StringHashEq::Eq, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > >> [no contained value], 
          constraint_is_extracted_ = std::vector<bool> of length 3, capacity 64 = {true, true, true}, objective_ = std::unique_ptr<operations_research::MPObjective> = {get() = {<No data fields>}}, 
          solution_hint_ = std::vector of length 0, capacity 0, time_limit_ = {rep_hi_ = 9223372036854775807, rep_lo_ = 4294967295}, construction_time_ = {rep_ = {rep_hi_ = 1633526157, rep_lo_ = 3602394716}}, num_threads_ = 1, 
          solver_specific_parameter_string_ = "", static global_count_mutex_ = {mu_ = {<std::__atomic_base<long>> = {static _S_alignment = 8, _M_i = 0}, static is_always_lock_free = true}}, static global_num_variables_ = 3, 
          static global_num_constraints_ = 3}
        infinity = inf
        x1 = 0x55555557a2d0
        x2 = 0x5555555749d0
        x3 = 0x5555555731c0
        objective = 0x5555555a06b0
        c0 = 0x555555581a20
        c1 = 0x55555557eda0
        c2 = 0x555555581d60
        result_status = operations_research::MPSolver::OPTIMAL
        activities = std::vector of length 0, capacity -4675
#7  0x00005555555596cb in operations_research::RunAllExamples () at /home/mizux/work/master/examples/cpp/linear_programming.cc:113
No locals.
#8  0x00005555555597f3 in main (argc=1, argv=0x7fffffffe1a8) at /home/mizux/work/master/examples/cpp/linear_programming.cc:125
No locals.
(gdb)
tkralphs commented 3 years ago

@tkralphs so if I understand you correctly, the current failing of addRows is expected so the current patch on stable/1.17 is OK.

Yes, the failure I see when running addRows with stand-along Clp stable/1.17 looks expected---there are comments in the code to this effect and you can verify that the code is (intentionally) buggy by inspection.

I've just do some test in or-tools (master branch) BUT targeting mizux/Clp stable/1.17 and we have issue when running sample (which was working until release/1.17.4) -> @tkralphs Should I deduce that our Clp wrapper is broken but luckily until 1.17.4 it was working ?

I'm not at all clear on this. The addRows example is fixed, but I don't know what that means for your simple test. I tried to eyeball the code and the stack trace above, but I can't really parse it too easily. It does seem strange that the seg fault seems to be coming on the call to matrix_->setDimensions(-1, numberColumns_);, which seems to indicate that matrix_ is not a valid pointer. Am I right? But anyway, if you can make a stand-alone example so we can easily replicate, that would help.

jjhforrest commented 3 years ago

I copied the latest stable versions of Clp and CoinUtils to Clp-1.17.4 and CoinUtils-2.11.4 to or-tools and got the seg fault. I have fixed in stable/1.17 and that works.

The tests failed in lp_test - but that seems to be SCIP I1007 09:07:33.801261 236851 lp_test.cc:189] ---- Mixed Integer programming example with SCIP ---- ... F1007 09:07:50.073259 236851 linear_solver.cc:285] Reduced cost only available for continuous problems

On Wed, Oct 6, 2021 at 8:43 PM Ted Ralphs @.***> wrote:

@tkralphs https://github.com/tkralphs so if I understand you correctly, the current failing of addRows is expected so the current patch on stable/1.17 is OK.

Yes, the failure I see when running addRows with stand-along Clp stable/1.17 looks expected---there are comments in the code to this effect and you can verify that the code is (intentionally) buggy by inspection.

I've just do some test in or-tools (master branch) BUT targeting mizux/Clp stable/1.17 and we have issue when running sample (which was working until release/1.17.4) -> @tkralphs https://github.com/tkralphs Should I deduce that our Clp wrapper is broken but luckily until 1.17.4 it was working ?

I'm not at all clear on this. The addRows example is fixed, but I don't know what that means for your simple test. I tried to eyeball the code and the stack trace above, but I can't really parse it too easily. It does seem strange that the seg fault seems to be coming on the call to matrix->setDimensions(-1, numberColumns);, which seems to indicate that matrix_ is not a valid pointer. Am I right? But anyway, if you can make a stand-alone example so we can easily replicate, that would help.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coin-or/Clp/issues/130#issuecomment-936993599, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWJYHAZHQJEFQOA5FHDW63UFSRHFANCNFSM4KQ42MTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

tkralphs commented 3 years ago

@Mizux Looks like this should fix. Can you verify? Then we can try to get a new release out.

Mizux commented 3 years ago

Sorry I get catch cold so won't be here for long BUT I just rebase mizux/Clp stable/1.17 to your tip then test against or-tools and it's work !

note: Once you'll create a release/1.17.7 tag I will create a branch stable/1.17.7 (and won't touch it anymore) while I will continue to rebase (and push -f) my stable/1.17 to keep to be upstream.

For the record

[0]─[~/work/master]
[*v*]─mizux@nuc10i7 %git diff -u
diff --git a/cmake/dependencies/CMakeLists.txt b/cmake/dependencies/CMakeLists.txt
index ddac17f99f..649cba5105 100644
--- a/cmake/dependencies/CMakeLists.txt
+++ b/cmake/dependencies/CMakeLists.txt
@@ -176,7 +176,7 @@ if(BUILD_Clp)
   FetchContent_Declare(
     Clp
     GIT_REPOSITORY "https://github.com/Mizux/Clp.git"
-    GIT_TAG "stable/1.17.4")
+    GIT_TAG "stable/1.17")
   FetchContent_MakeAvailable(Clp)
   list(POP_BACK CMAKE_MESSAGE_INDENT)
   message(CHECK_PASS "fetched")

see: https://github.com/Mizux/Clp/commits/stable/1.17

Trace of the run of ortools/linear_solver/samples/linear_programming.cc from google/or-tools (master)

[0]─[~/work/master/build]
[^u^]─mizux@nuc10i7 %gdb bin/linear_programming
GNU gdb (GDB) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bin/linear_programming...
(No debugging symbols found in bin/linear_programming)
(gdb) run
Starting program: /home/mizux/work/stable/build/bin/linear_programming 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
I1007 20:58:33.839479 272248 linear_programming.cc:30] ---- Linear programming example with GLOP ----
I1007 20:58:33.840005 272248 linear_programming.cc:77] Number of variables = 3
I1007 20:58:33.840018 272248 linear_programming.cc:78] Number of constraints = 3
I1007 20:58:33.840512 272248 linear_programming.cc:87] Problem solved in 0 milliseconds
I1007 20:58:33.840522 272248 linear_programming.cc:90] Optimal objective value = 733.333
I1007 20:58:33.840532 272248 linear_programming.cc:93] x1 = 33.3333
I1007 20:58:33.840536 272248 linear_programming.cc:94] x2 = 66.6667
I1007 20:58:33.840538 272248 linear_programming.cc:95] x3 = 0
I1007 20:58:33.840541 272248 linear_programming.cc:97] Advanced usage:
I1007 20:58:33.840543 272248 linear_programming.cc:98] Problem solved in 2 iterations
I1007 20:58:33.840546 272248 linear_programming.cc:99] x1: reduced cost = 0
I1007 20:58:33.840548 272248 linear_programming.cc:100] x2: reduced cost = 0
I1007 20:58:33.840551 272248 linear_programming.cc:101] x3: reduced cost = -2.66667
I1007 20:58:33.840554 272248 linear_programming.cc:103] c0: dual value = 3.33333 activity = 100
I1007 20:58:33.840557 272248 linear_programming.cc:105] c1: dual value = 0.666667 activity = 600
I1007 20:58:33.840560 272248 linear_programming.cc:107] c2: dual value = -0 activity = 200
I1007 20:58:33.840583 272248 linear_programming.cc:30] ---- Linear programming example with CLP ----
I1007 20:58:33.840678 272248 linear_programming.cc:77] Number of variables = 3
I1007 20:58:33.840682 272248 linear_programming.cc:78] Number of constraints = 3
I1007 20:58:33.841034 272248 linear_programming.cc:87] Problem solved in 0 milliseconds
I1007 20:58:33.841041 272248 linear_programming.cc:90] Optimal objective value = 733.333
I1007 20:58:33.841045 272248 linear_programming.cc:93] x1 = 33.3333
I1007 20:58:33.841048 272248 linear_programming.cc:94] x2 = 66.6667
I1007 20:58:33.841050 272248 linear_programming.cc:95] x3 = 0
I1007 20:58:33.841052 272248 linear_programming.cc:97] Advanced usage:
I1007 20:58:33.841054 272248 linear_programming.cc:98] Problem solved in 2 iterations
I1007 20:58:33.841056 272248 linear_programming.cc:99] x1: reduced cost = -8.88178e-16
I1007 20:58:33.841058 272248 linear_programming.cc:100] x2: reduced cost = 0
I1007 20:58:33.841061 272248 linear_programming.cc:101] x3: reduced cost = -2.66667
I1007 20:58:33.841063 272248 linear_programming.cc:103] c0: dual value = 3.33333 activity = 100
I1007 20:58:33.841066 272248 linear_programming.cc:105] c1: dual value = 0.666667 activity = 600
I1007 20:58:33.841069 272248 linear_programming.cc:107] c2: dual value = -0 activity = 200
I1007 20:58:33.841099 272248 linear_programming.cc:30] ---- Linear programming example with GUROBI_LP ----
I1007 20:58:33.841178 272248 linear_programming.cc:38] Supports for solver GUROBI_LP not linked in.
I1007 20:58:33.841182 272248 linear_programming.cc:30] ---- Linear programming example with CPLEX_LP ----
I1007 20:58:33.841184 272248 linear_programming.cc:38] Supports for solver CPLEX_LP not linked in.
I1007 20:58:33.841186 272248 linear_programming.cc:30] ---- Linear programming example with GLPK_LP ----
I1007 20:58:33.841189 272248 linear_programming.cc:38] Supports for solver GLPK_LP not linked in.
I1007 20:58:33.841190 272248 linear_programming.cc:30] ---- Linear programming example with XPRESS_LP ----
I1007 20:58:33.841193 272248 linear_programming.cc:38] Supports for solver XPRESS_LP not linked in.
[Inferior 1 (process 272248) exited normally]

So good job coin-or dev ! -> you can close this issue ps: if you can ping me once release/1.17.7 is out ;)