search

google / or-tools

Google's Operations Research tools:
https://developers.google.com/optimization/
Apache License 2.0
11.25k stars 2.13k forks source link

Bump SCIP 6.0.0 -> 6.0.1 #1115

Closed Mizux closed 5 years ago

Mizux commented 5 years ago

Need to test the last SCIP 6.0.1 and update doc and code accordingly

SCIP Install

I'll only test the CMake based build of SCIP using the command: SCIP download link: https://scip.zib.de/download.php?fname=scipoptsuite-6.0.1.tgz
SCIP install doc: https://scip.zib.de/doc/html/INSTALL.php

Unix:

cmake -H. -Bbuild -DSHARED=OFF -DGMP=OFF -DGCG=OFF -DSYM=none -DIPOPT=OFF -DREADLINE=OFF -DZIMPL=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_INSTALL_PREFIX=$HOME/scip
cmake --build build --target install
export UNIX_SCIP_DIR=$HOME/scip
cd <ortools_root_path>
make clean
make cc
make run SOURCE=examples/test/lp_test.cc
make python
make run SOURCE=examples/test/lp_test.py

Windows:

Tasks

Here the tasks to do to test SCIP (sciptoptsuite) 6.0.1

note: currently SCIP is not supported in CMake-based build or Bazel-based build (PR welcome, or i'll do it later)

need to create a report like this: https://github.com/google/or-tools/issues/948#issuecomment-442447342

Related Issue

1188 SCIP_interface use macros not available publicly

1023 MacOS python package with SCIP support (in this case we should use SCIP static lib)

Annexe

Mizux commented 5 years ago

trying to build SCIP using:

FROM ubuntu:18.10
LABEL maintainer="corentinl@google.com"

RUN apt update -qq \
&& apt install -yq git build-essential zlib1g-dev cmake \
lsb-release pkg-config autoconf autotools-dev \
&& apt clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

WORKDIR /root

# Copy the scip archive
COPY scipoptsuite-6.0.1.tgz /root/scipoptsuite.tgz

RUN cd /tmp \
&& tar xzvf /root/scipoptsuite.tgz \
&& cd scipoptsuite-6.0.1 \
&& cmake -H. -Bbuild -DSHARED=OFF -DGMP=OFF -DGCG=OFF -DCMAKE_INSTALL_PREFIX=/scip \
&& cmake --build build \
&& cmake --build build --target install \
&& cd ..
&& rm -rf scip*

-> got libsoplex-pic.a but libscip.a

$ export UNIX_SCIP_DIR=$HOME/scip
$ make clean_cc
$ make cc
...
/usr/bin/ld: /usr/local/google/home/corentinl/scip/lib/libscip.a(cons_linear.c.o): relocation R_X86_64_PC32 against symbol `SCIPincludeConshdlrLinear' can not be used when making a shared object; recompile with -fPIC
Mizux commented 5 years ago

Seems to be fixed by:

cmake -H. -Bbuild -DSHARED=OFF -DGMP=OFF -DGCG=OFF -DSYM=none -DIPOPT=OFF -DREADLINE=OFF -DZIMPL=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_INSTALL_PREFIX=$HOME/scip

notice the -DCMAKE_POSITION_INDEPENDENT_CODE=ON

Mizux commented 5 years ago

SCIP 6.0.1 CMake based build with SHARED=OFF seems broken on windows :disappointed: I was not able to install SCIP 6.0.1 on my windows VM, (needed to test the integration against google/or-tools)

Windows

Environment

Protocol

  1. Download SCIPoptsuite and Extract it.

    tar xzvf scipoptsuite-6.0.1.tgz
cd scipoptsuite-6.0.1

  2. CMake configure with:

    cmake -H. -Bbuild -DSHARED=OFF -DGMP=OFF -DGCG=OFF -DSYM=none -DIPOPT=OFF -DREADLINE=OFF -DZIMPL=OFF -DCMAKE_INSTALL_PREFIX=%userprofile%\scip -G "Visual Studio 15 2017 Win64"
...
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/.../scipoptsuite-6.0.1/build

  3. CMake build using:

    cmake --build build --config Release

  4. CMake install using:

    cmake --build build --target install --config Release

Observed

Problem

Once dependencies issue is fixed, there is an other issue. :disappointed: 

...
Project "C:\Users\...\Downloads\scipoptsuite-6.0.1\build\ALL_BUILD.vcxproj" (1) is building "C:\Users\...\Downloads\scipoptsuite-6.0.1\build\scip\src\scip.vcxproj" (8) on node 1 (default
targets).
PrepareForBuild:
  Creating directory "scip.dir\Release\".
  Creating directory "C:\Users\...\Downloads\scipoptsuite-6.0.1\build\bin\Release\".
  Creating directory "scip.dir\Release\scip.tlog\".
InitializeBuildStatus:
  Creating "scip.dir\Release\scip.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
CustomBuild:
  Building Custom Rule C:/Users/.../Downloads/scipoptsuite-6.0.1/scip/src/CMakeLists.txt
  CMake does not need to re-run because C:/Users/.../Downloads/scipoptsuite-6.0.1/build/scip/src/CMakeFiles/generate.stamp is up-to-date.
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\CL.exe /c /I"C:\Users\...\Downloads\scipoptsuite-6.0.1\soplex\src" /I"C:\Users\coren
  tinl\Downloads\scipoptsuite-6.0.1\build" /I"C:\Users\...\Downloads\scipoptsuite-6.0.1\scip\src" /nologo /W3 /WX- /diagnostics:classic /O2 /Ob2 /D WIN32 /D _WINDOWS /D NDEBUG /D EXTERN=extern
  /D _CRT_SECURE_NO_WARNINGS /D "CMAKE_INTDIR=\"Release\"" /D _MBCS /Gm- /EHsc /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"scip.dir\Release\\" /Fd"scip.dir\Release\vc141.pdb" /Gd
  /TC /wd4100 /wd4244 /FC /errorReport:queue "C:\Users\...\Downloads\scipoptsuite-6.0.1\scip\src\main.c"
  main.c
Link:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\link.exe /ERRORREPORT:QUEUE /OUT:"C:\Users\...\Downloads\scipoptsuite-6.0.1\build\bi
  n\Release\scip.exe" /INCREMENTAL:NO /NOLOGO ..\..\lib\Release\scip.lib ..\..\lib\Release\soplex.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg
  32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"C:/Users/.../Downloads/scipoptsuite-6.0.1/build/bin/Release/scip.pdb" /SUBSYSTEM:CONSOLE
  /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/Users/.../Downloads/scipoptsuite-6.0.1/build/lib/Release/scip.lib" /MACHINE:X64  /machine:x64 scip.dir\Release\main.obj
     Creating library C:/Users/.../Downloads/scipoptsuite-6.0.1/build/lib/Release/scip.lib and object C:/Users/.../Downloads/scipoptsuite-6.0.1/build/lib/Release/scip.exp
..\..\lib\Release\scip.lib : fatal error LNK1106: invalid file or disk full: cannot seek to 0x6B00F8 [C:\Users\...\Downloads\scipoptsuite-6.0.1\build\scip\src\scip.vcxproj]
Done Building Project "C:\Users\...\Downloads\scipoptsuite-6.0.1\build\scip\src\scip.vcxproj" (default targets) -- FAILED.

"C:\Users\...\Downloads\scipoptsuite-6.0.1\build\ALL_BUILD.vcxproj" (default target) (1) ->
"C:\Users\...\Downloads\scipoptsuite-6.0.1\build\scip\src\scip.vcxproj" (default target) (8) ->
(Link target) ->
  ..\..\lib\Release\scip.lib : fatal error LNK1106: invalid file or disk full: cannot seek to 0x6B00F8 [C:\Users\...\Downloads\scipoptsuite-6.0.1\build\scip\src\scip.vcxproj]

    112 Warning(s)
    1 Error(s)

Analyze

note: we can see in the trace that CMake try to build scip.exe binary before the libscip.lib

Solution (Dirty quick fix)

We could rename the target scip e.g. scip_solver or whatever. note: Need to do the change in scip/src/CMakeLists.txt AND scip/check/CMakeLists.txt (use of generator expression $<TARGET_FILE:scip>)

Status

Once both problems solved, integration with OR-Tools (Makefile based build) seems OK.

  1. Build Or-Tools

    make clean_cc
set WINDOWS_SCIP_DIR=%userprofile%\scip
make detect_cc # to check USE_SCIP etc...
make cc

  2. Build/test an example

    make run SOURCE=examples\tests\lp_test.cc
...
Mizux commented 5 years ago

WARNING: DEPRECATED the patch

diff --git a/scip/check/CMakeLists.txt b/scip/check/CMakeLists.txt
index d48e8e9..6aa6981 100644
--- a/scip/check/CMakeLists.txt
+++ b/scip/check/CMakeLists.txt
@@ -5,7 +5,7 @@ include(CTest)
 #
 add_custom_target(scip_check
                 COMMAND ${CMAKE_CTEST_COMMAND} -R "-default" -E "applications"                                                                                                                                                                    --output-on-failure
-                DEPENDS scip
+                DEPENDS scip_solver
                 )

 #
@@ -390,7 +390,7 @@ macro(add_instancetests instances settings prefix)
             # treat the instance as a tuple (list) of two values
             #
             add_test(NAME ${prefix}-${setting}-${basename}
-                    COMMAND $<TARGET_FILE:scip> -f ${PROJECT_SOURCE_DIR}/check/                                                                                                                                                                   ${path} -s ${PROJECT_SOURCE_DIR}/coverage/settings/${setting}.set -o ${optval} $                                                                                                                                                                   {optval}
+                    COMMAND $<TARGET_FILE:scip_solver> -f ${PROJECT_SOURCE_DIR}                                                                                                                                                                   /check/${path} -s ${PROJECT_SOURCE_DIR}/coverage/settings/${setting}.set -o ${op                                                                                                                                                                   tval} ${optval}
                     )
             set_tests_properties(${prefix}-${setting}-${basename}
                                 PROPERTIES
@@ -429,7 +429,7 @@ foreach(instance ${instances_MINLP})
     #
     configure_file(interactiveshell/solutiontest.bat.in interactiveshell/soluti                                                                                                                                                                   ontest-${basename}.bat)
         add_test(NAME MINLP-solutiontest-${basename}
-                COMMAND $<TARGET_FILE:scip> -b ${PROJECT_BINARY_DIR}/check/inte                                                                                                                                                                   ractiveshell/solutiontest-${basename}.bat
+                COMMAND $<TARGET_FILE:scip_solver> -b ${PROJECT_BINARY_DIR}/che                                                                                                                                                                   ck/interactiveshell/solutiontest-${basename}.bat
                 )
         set_tests_properties(MINLP-solutiontest-${basename}
                             PROPERTIES
@@ -481,7 +481,7 @@ foreach(instance ${instances_MIP})
         endif()

         add_test(NAME MIP-readertest-${extension}-${basename}
-                COMMAND $<TARGET_FILE:scip> -b ${PROJECT_BINARY_DIR}/check/inte                                                                                                                                                                   ractiveshell/readertest-${extension}-${basename}.bat
+                COMMAND $<TARGET_FILE:scip_solver> -b ${PROJECT_BINARY_DIR}/che                                                                                                                                                                   ck/interactiveshell/readertest-${extension}-${basename}.bat
                 )
         set_tests_properties(MIP-readertest-${extension}-${basename}
                             PROPERTIES
@@ -503,7 +503,7 @@ if(NOT ${TPI} STREQUAL "none")
         #
         configure_file(interactiveshell/concurrentsolve.bat.in interactiveshell                                                                                                                                                                   /concurrentsolve-${basename}.bat)
         add_test(NAME MIP-concurrentsolve-${basename}
-                COMMAND $<TARGET_FILE:scip> -b ${PROJECT_BINARY_DIR}/check/inte                                                                                                                                                                   ractiveshell/concurrentsolve-${basename}.bat
+                COMMAND $<TARGET_FILE:scip_solver> -b ${PROJECT_BINARY_DIR}/che                                                                                                                                                                   ck/interactiveshell/concurrentsolve-${basename}.bat
                 )
         set_tests_properties(MIP-concurrentsolve-${basename}
                             PROPERTIES
@@ -531,7 +531,7 @@ foreach(instance ${instances_MIP})
         # add test that executes the configured batch file and greps the valida                                                                                                                                                                   tion expression
         #
         add_test(NAME MIP-mipstart-${basename}
-                COMMAND $<TARGET_FILE:scip> -b interactiveshell/MIP-mipstart-${                                                                                                                                                                   basename}.bat
:
ck/interactiveshell/readertest-${extension}-${basename}.bat
                 )
         set_tests_properties(MIP-readertest-${extension}-${basename}
                             PROPERTIES
@@ -503,7 +503,7 @@ if(NOT ${TPI} STREQUAL "none")
         #
         configure_file(interactiveshell/concurrentsolve.bat.in interactiveshell/concurrentsolve-${basename}.bat)
         add_test(NAME MIP-concurrentsolve-${basename}
-                COMMAND $<TARGET_FILE:scip> -b ${PROJECT_BINARY_DIR}/check/interactiveshell/concurrentsolve-${basename}.bat
+                COMMAND $<TARGET_FILE:scip_solver> -b ${PROJECT_BINARY_DIR}/check/interactiveshell/concurrentsolve-${basename}.bat
                 )
         set_tests_properties(MIP-concurrentsolve-${basename}
                             PROPERTIES
@@ -531,7 +531,7 @@ foreach(instance ${instances_MIP})
         # add test that executes the configured batch file and greps the validation expression
         #
         add_test(NAME MIP-mipstart-${basename}
-                COMMAND $<TARGET_FILE:scip> -b interactiveshell/MIP-mipstart-${basename}.bat
+                COMMAND $<TARGET_FILE:scip_solver> -b interactiveshell/MIP-mipstart-${basename}.bat
                 )
         set_tests_properties(MIP-mipstart-${basename}
                             PROPERTIES
@@ -554,7 +554,7 @@ file(MAKE_DIRECTORY ${shell_tmp_dir})
 configure_file(interactiveshell/interactiveshell.bat.in interactiveshell/interactiveshell.bat)

 add_test(NAME interactiveshell
-        COMMAND $<TARGET_FILE:scip> -b ${PROJECT_BINARY_DIR}/check/interactiveshell/interactiveshell.bat
+        COMMAND $<TARGET_FILE:scip_solver> -b ${PROJECT_BINARY_DIR}/check/interactiveshell/interactiveshell.bat
         )
 set_tests_properties(interactiveshell
                     PROPERTIES
@@ -563,7 +563,7 @@ set_tests_properties(interactiveshell
                     )

 add_test(NAME write_and_read_settings
-        COMMAND $<TARGET_FILE:scip> -b ${CMAKE_CURRENT_SOURCE_DIR}/interactiveshell/write_and_read_settings.bat
+        COMMAND $<TARGET_FILE:scip_solver> -b ${CMAKE_CURRENT_SOURCE_DIR}/interactiveshell/write_and_read_settings.bat
         )
 set_tests_properties(write_and_read_settings
                     PROPERTIES
@@ -677,7 +677,7 @@ foreach(instance ${instances_MIP})
     if(NOT linclassfound EQUAL -1 )
         configure_file(interactiveshell/linclass.bat.in interactiveshell/linclass-${basename}.bat)
         add_test(NAME MIP-linclass-${basename}
-                COMMAND $<TARGET_FILE:scip> -b interactiveshell/linclass-${basename}.bat
+                COMMAND $<TARGET_FILE:scip_solver> -b interactiveshell/linclass-${basename}.bat
                 )
         set_tests_properties(MIP-linclass-${basename}
                             PROPERTIES
@@ -708,7 +708,7 @@ if (UNIX)
     endif (APPLE)
     configure_file(interactiveshell/signal-handling-sigterm.bat.in interactiveshell/signal-handling-sigterm.bat)
     add_test(NAME signal-handling-sigterm
-            COMMAND ${timeoutcommand} -sSIGTERM 1 $<TARGET_FILE:scip> -b interactiveshell/signal-handling-sigterm.bat
+            COMMAND ${timeoutcommand} -sSIGTERM 1 $<TARGET_FILE:scip_solver> -b interactiveshell/signal-handling-sigterm.bat
             )
     set_tests_properties(signal-handling-sigterm
                         PROPERTIES
diff --git a/scip/src/CMakeLists.txt b/scip/src/CMakeLists.txt
index 873b869..5a51889 100644
--- a/scip/src/CMakeLists.txt
+++ b/scip/src/CMakeLists.txt
@@ -938,25 +938,25 @@ target_link_libraries(libscip PRIVATE

 if(SHARED)
     target_link_libraries(libscip PRIVATE ${ZIMPL_PIC_LIBRARIES} ${LPS_PIC_LIBRARIES} ${SYM_PIC_LIBRARIES})
-    add_executable(scip main.c ${scipsources} ${objscipsources} ${nlpisources} ${lpisources} ${tpisources} ${symsources})
+    add_executable(scip_solver main.c ${scipsources} ${objscipsources} ${nlpisources} ${lpisources} ${tpisources} ${symsources})
 else()
     target_link_libraries(libscip PRIVATE ${ZIMPL_LIBRARIES} ${LPS_LIBRARIES} ${SYM_LIBRARIES})
-    add_executable(scip main.c)
-    target_link_libraries(scip libscip)
+    add_executable(scip_solver main.c)
+    target_link_libraries(scip_solver libscip)
 endif()

 if(CMAKE_BUILD_TYPE EQUAL "Debug")
     find_package(Sanitizers)
-    add_sanitizers(scip)
+    add_sanitizers(scip_solver)
 endif()

-target_compile_definitions(scip PRIVATE EXTERN=extern)
+target_compile_definitions(scip_solver PRIVATE EXTERN=extern)

-target_link_libraries(scip ${ZLIB_LIBRARIES} ${Readline_LIBRARY} ${GMP_LIBRARIES}
+target_link_libraries(scip_solver ${ZLIB_LIBRARIES} ${Readline_LIBRARY} ${GMP_LIBRARIES}
    ${ZIMPL_LIBRARIES} ${LPS_LIBRARIES} ${SYM_LIBRARIES} ${THREAD_LIBRARIES} ${NLPI_LIBRARIES})

 add_dependencies(libscip scip_update_githash)
-add_dependencies(scip scip_update_githash)
+add_dependencies(scip_solver scip_update_githash)

 set_target_properties(libscip PROPERTIES
     VERSION ${SCIP_VERSION_MAJOR}.${SCIP_VERSION_MINOR}.${SCIP_VERSION_PATCH}.${SCIP_VERSION_SUB}
@@ -964,7 +964,7 @@ set_target_properties(libscip PROPERTIES
     INSTALL_RPATH_USE_LINK_PATH TRUE)

 # set the install rpath to the installed destination
-set_target_properties(scip PROPERTIES
+set_target_properties(scip_solver PROPERTIES
     INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib"
     INSTALL_RPATH_USE_LINK_PATH TRUE)

@@ -982,14 +982,14 @@ install(FILES ${symheaders} DESTINATION include/symmetry)
 install(FILES ${blockmemshellheader} DESTINATION include/blockmemshell)

 # install the binary and the library to appropriate lcoations and add them to an export group
-install(TARGETS scip libscip EXPORT scip-targets
+install(TARGETS scip_solver libscip EXPORT scip-targets
         LIBRARY DESTINATION lib
         ARCHIVE DESTINATION lib
         RUNTIME DESTINATION bin
         INCLUDES DESTINATION include)

 # Add all targets to the build-tree export set
-export(TARGETS scip libscip
+export(TARGETS scip_solver libscip
   FILE "${CMAKE_BINARY_DIR}/scip-targets.cmake")

 #make soplex and zimpl dir absolute for the config file
Mizux commented 5 years ago

A Better patch, simply change the /IMPLIB name of the scip TARGET:

--- a/scip/src/CMakeLists.txt
+++ b/scip/src/CMakeLists.txt
@@ -942,6 +942,7 @@ if(SHARED)
 else()
     target_link_libraries(libscip PRIVATE ${ZIMPL_LIBRARIES} ${LPS_LIBRARIES} ${SYM_LIBRARIES})
     add_executable(scip main.c)
+    set_target_properties(scip PROPERTIES IMPORT_SUFFIX ".exe.lib")
     target_link_libraries(scip libscip)
 endif()

note: Patch send to the SCIP (CMake support) Team...

rlloretb commented 5 years ago

Don't we need to do make third_party on Unix then, according to OP?

rlloretb commented 5 years ago

I'm having this error on MacOS, python:

error: can't combine user with prefix, execprefix/home, or install(plat)base make: *** [makefiles/Makefile.python.mk:1116: install_python] Error 1

lperron commented 5 years ago

This is a known error on some Linux distro. What is your distro ?

Le lun. 10 juin 2019 à 12:58, Roger Lloret-Batlle notifications@github.com a écrit :

I'm having this error on MacOS, python:

error: can't combine user with prefix, execprefix/home, or install(plat)base make: *** [makefiles/Makefile.python.mk:1116: install_python] Error 1

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/or-tools/issues/1115?email_source=notifications&email_token=ACUPL3K2SATIFVYHHY2AWF3PZYXUHA5CNFSM4G4TXB22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXJSIDQ#issuecomment-500376590, or mute the thread https://github.com/notifications/unsubscribe-auth/ACUPL3KS52JU2FGKPAN44UDPZYXUHANCNFSM4G4TXB2Q .

rlloretb commented 5 years ago

I am on MacOS 10.12.6

By the way, does or-tools 7.1 work with SCIP 6.0.0 or SCIP 6.0.1?