Installation complete but tests not completing/freezing

banjodd commented 11 months ago

Hello, I'm trying to use the scip/MATLAB interface but I get this issue where, after the installation process, trying to solve whatever problem using scip solver is impossible due to the fact that MATLAB completely freezes when I try to run one (quit forcing is the only way to restart the application). In fact, even the example cases that the installation script suggests that I should run are not able to complete and freezes the app. Can anybody help?

Windows 11 Scip opt suite: 8.0.4 MATLAB version 2023a

pfetsch commented 11 months ago

Can you please check whether https://www.mathworks.com/help/matlab/matlab_external/invalid-mex-file-error.html helps?

banjodd commented 11 months ago

I used the Dependency walker as specified and managed to save this .dwi file:

MATLAB.zip

pfetsch commented 11 months ago

I had a look, but cannot get any useful information out of the file. I am afraid that I cannot help you in this case, but I will try to reproduce it. Can you maybe post the output of the installation process?

tinusd commented 8 months ago

Hi there

I am having a similar problem. I am using SCIP suite 8.1.0, Matlab 2020b. I do the following:

>> matlabSCIPInterface_install

Searching for SCIP install ...
Checking environment variables SCIPOPTDIR and SCIPDIR ...
SCIPOPTDIR variable found.

Setting SCIP directory to 'c:/Program Files/SCIPOptSuite 8.1.0/'.

SCIP install using 'make install' or '.exe' detected.
Was the specified SCIP library build using cmake/.exe? (y/n) y

------------------------------------------------
SCIP MEX FILE INSTALL

MEX Call:
mex  -largeArrayDims   -DNO_CONFIG_HEADER scip/scipmex.cpp scip/scipeventmex.cpp scip/scipnlmex.cpp -Iscip/Include -I'c:/Program Files/SCIPOptSuite 8.1.0//include' -I'c:/Program Files/SCIPOptSuite 8.1.0//include/blockmemshell' -Iopti  -IInclude -L'c:/Program Files/SCIPOptSuite 8.1.0//lib/' -lscip  -L'C:\\Program Files\\MATLAB\\R2020b\extern\lib\win64\mingw64' -lut  -output scip -DML_VER=R2020b -DOPTI_VER=3.00

Building with 'MinGW64 Compiler (C++)'.
MEX completed successfully.
Done!

------------------------------------------------
INSTALLING Matlab-SCIP Interface ...

- Checking MATLAB version and operating system ...
MATLAB (R2020b) 64bit (Windows x64) detected.

- Checking MEX file release information ...
MEX files match interface release.

Would you like to run post installation tests? (Recommended) (y/n):

Once I say yes to the post installation tests, it just hangs at

Checking OPTI Toolbox Installation:
Checking    LP Solver Results...  scip  1

Any idea what could be going wrong?

pfetsch commented 8 months ago

Do you use IPOPT? Then possibly the problems as described in the README affect you (see "Troubleshooting").

tinusd commented 8 months ago

Thanks for the quick reply.

Are you refering to the Linux: Problems with Lapack/MKL using SCIP with IPOPT section? I am using Windows (forgot to mention), so would it still be relevant? Also, would IPOPT be using during the LP solver tests? [Also, when I used the OPTI toolbox, it did pass the LP solver tests (and all others that were available), but it wasn't using SCIP]

tinusd commented 8 months ago

I saw that you have updated the repo. However, I am now getting the following:

>> matlabSCIPInterface_install

Searching for SCIP install ...
Checking environment variables SCIPOPTDIR and SCIPDIR ...
SCIPOPTDIR variable found.

Setting SCIP directory to 'c:/Program Files/SCIPOptSuite 8.1.0/'.

SCIP install using 'make install' or '.exe' detected.

------------------------------------------------
SCIP MEX FILE INSTALL

MEX Call:
mex  -largeArrayDims scip/scipmex.cpp scip/scipeventmex.cpp scip/scipnlmex.cpp -Iscip/Include -I'c:/Program Files/SCIPOptSuite 8.1.0//include' -I'c:/Program Files/SCIPOptSuite 8.1.0//include/blockmemshell' -Iopti  -IInclude -L'c:/Program Files/SCIPOptSuite 8.1.0//lib/' -lscip  -L'C:\\Program Files\\MATLAB\\R2020b\extern\lib\win64\mingw64' -lut  -output scip -DML_VER=R2020b -DOPTI_VER=3.00

Building with 'MinGW64 Compiler (C++)'.
C:\Users\<XXX>\work\opti\MatlabSCIPInterface\Source\scip\scipnlmex.cpp: In function 'double addNonlinearCon(SCIP*, SCIP_VAR**, double*, size_t, double, double, double*, size_t, bool)':
C:\Users\<XXX>\work\opti\MatlabSCIPInterface\Source\scip\scipnlmex.cpp:69:8: warning: 'double addNonlinearCon(SCIP*, SCIP_VAR**, double*, size_t, double, double, double*, size_t, bool)' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
 double addNonlinearCon(
        ^~~~~~~~~~~~~~~

Done!
Error Compiling SCIP!
C:\Users\<XXX>\AppData\Local\Temp\mex_168163204913956_17832\scipmex.obj:scipmex.cpp:(.text+0x39b): undefined reference to `__imp__Z25SCIPincludeCtrlCEventHdlrP4Scip'
C:\Users\<XXX>\AppData\Local\Temp\mex_168163204913956_17832\scipmex.obj:scipmex.cpp:(.text+0x24a5): undefined reference to `__imp__Z15addNonlinearConP4ScipPP8SCIP_VarPdyddS4_yb'
C:\Users\<XXX>\AppData\Local\Temp\mex_168163204913956_17832\scipmex.obj:scipmex.cpp:(.text+0x2547): undefined reference to `__imp__Z15addNonlinearConP4ScipPP8SCIP_VarPdyddS4_yb'
C:\Users\<XXX>\AppData\Local\Temp\mex_168163204913956_17832\scipmex.obj:scipmex.cpp:(.text+0x4711): undefined reference to `__imp__Z15addNonlinearConP4ScipPP8SCIP_VarPdyddS4_yb'
collect2.exe: error: ld returned 1 exit status

------------------------------------------------
INSTALLING Matlab-SCIP Interface ...

- Checking MATLAB version and operating system ...
MATLAB (R2020b) 64bit (Windows x64) detected.

- Checking MEX file release information ...
MEX file is not compatible with this version of the Interface.
Error using matlabSCIPInterface_install>mexFileCheck (line 382)
MEX file incompatible.

Error in matlabSCIPInterface_install (line 88)
if (~mexFileCheck(localVer))

Any idea what is going wrong? Help would be greatly appreciated.

tinusd commented 8 months ago

It seems expre = [expre ' -DNO_CONFIG_HEADER']; was missing in the setLibrary function; it then compiles SCIP. However, I am still sitting with the problem where I am just stuck at the LP solver test...

pfetsch commented 8 months ago

Sorry for the problems - some comments that may help:

I have updated the installation scripts to be compatible with the upcoming SCIP 9 and thought it would work for SCIP 8.1, but I seemed to be wrong. So one obviously still needs the -DNO_CONFIG_HEADER option for SCIP 8.1 (but not for SCIP 9).
Unfortunately, I have no clue what the problem is and I am not sure whether you use IPOPT. If you do, my guess is that it has to do with the Lapack/MKL problem that I mentioned above. IPOPT and SCIP use Lapack and Matlab just replaces the corresponding library by the MKL (at least under Linux and I guess also under Windows). But the MKL uses 64 bit integers, so there is a size difference, which could case the problem. As explained in the Readme I unfortunately do not know an easy fix. Can you please try without IPOPT (if you use it) and check whether the problem still appears?
I am not sure how the OPTItoolbox resolved this issue, but at that time it may be that IPOPT was not used for SCIP.

tinusd commented 8 months ago

Thanks for your comments. I see. Dumb question though, but what is the easiest way not to use IPOPT? It is bundled with SCIP.

(As a side note, I used the Python interface and it worked. So it could indeed be Matlab.)

pfetsch commented 8 months ago

You would need to build SCIP from the source code. With cmake you would write cmake ... -DIPOPT=off with make you would use make ... IPOPT=false.

tinusd commented 8 months ago

I will do so. Please note, I am not the most experienced in building, so it may take a while until I get back to you.

pfetsch commented 8 months ago

Good luck and sorry for the hassle. I hope that this will resolve the issue.

tinusd commented 8 months ago

So, I managed to build SCIP from the source code, but I had to exclude quite a few libraries (I believe it was about 3, excluding IPOPT). Here it is:

scip -v

AMPL/SCIP Optimizer [8.1.0] (Windows AMD64), driver(20240121), MP(20240115)
External libraries:
Soplex 6.0.4 Linear Programming Solver developed at Zuse Institute Berlin (soplex.zib.de) [GitHash: 950b165-]
CppAD 20180000.0 Algorithmic Differentiation of C++ algorithms developed by B. Bell (github.com/coin-or/CppAD)
bliss 0.77 Computing Graph Automorphisms by T. Junttila and P. Kaski (users.aalto.fi/~tjunttil/bliss/)
sassy 1.1 Symmetry preprocessor by Markus Anders (github.com/markusa4/sassy)

I then created a new folder for "my" SCIP, and added the bin, include, and lib folders. I must add, I copied the include folder from the already compiled SCIP that I downloaded, since I could not find it in my build.

Unfortunately, I am still getting the same problem, where SCIP just hangs at the LP solver. I made sure that I updated all the environmental variables.

pfetsch commented 8 months ago

We are currently trying to reproduce the problem on a virtual machine.

tinusd commented 8 months ago

Thank you; I hope you manage to reproduce it. I always tried to use it on another laptop using Octave, but it crashed when doing the LP solver test.

pfetsch commented 8 months ago

I have tried to implement a work around. You can try whether it works, but I have not been able to test it myself.

tinusd commented 8 months ago

So you did manage to reproduce the problem then? What was the problem?

And, btw, it works!:

Checking OPTI Toolbox Installation:
Checking    LP Solver Results...  scip  1  2  3  4  5 Ok
Checking  MILP Solver Results...  scip  1  2  3  4  5 Ok
Checking    QP Solver Results...  scip  1  2  3 Ok
Checking  MIQP Solver Results...  scip  1  2 Ok
Checking   NLP Solver Results...  scip  1  2 Ok
Checking MINLP Solver Results...  scip  1 Ok

Toolbox checked out ok! - Enjoy.

Thanks a lot :)

pfetsch commented 8 months ago

We actually have no idea what the problem is. We just build the debug version of the code. @nsimon-TUDa found this solution by chance.

pfetsch commented 8 months ago

I am closing this issue for now (possibly we find the reason in the future).

scipopt / MatlabSCIPInterface

Installation complete but tests not completing/freezing #7