ubccr / software-layer

CCR Software Layer
GNU General Public License v2.0
3 stars 6 forks source link

update Gurobi software to version 10.0.1 #41

Closed edasdemirlab closed 1 year ago

edasdemirlab commented 1 year ago

Hello,

We plan to conduct a wide range of experiments using the Gurobi optimizer through Python. The latest version available in CCR is 9.1.1, which was released in 2020 and is outdated, as it is no longer fully supported by Gurobi. Furthermore, the latest version, 10.0.1, provides impressive performance improvements compared to 9.x.x versions.

I reached out to the Easy Build community for help in creating a recipe for Gurobi version 10.0.1, and I was fortunate to learn that they recently opened a pull request for this version a few days ago

(PR #17847 - https://github.com/easybuilders/easybuild-easyconfigs/pull/17847).

They provided me with some installation instructions, but I do not have a sufficient background in this area and am unsure how to proceed.

Here are the instructions they provided:

"grape" said:

Please read the comments in the PR regarding the license file! You may need to adjust it. Otherwise, if you run eb --from-pr 17847, you should be able to install it. You may also need to add the --robot flag.

"Kenneth Hoste (boegel)" said:

@Erdi mentioned that they need an easyconfig for Gurobi 10.0.1 with the GCC they have already installed, so a simple --from-pr won’t work. However, they can use the easyconfig file that is available in your PR and modify the toolchain and dependencies to match what they already have. Here is a starting point to get a local copy of the easyconfig file: eb --from-pr 17847 --copy-ec Gurobi-10.0.1.eb

To conclude, we were wondering if there is any possibility that you obtain and install the latest version 10.0.1 on CCR? Please let me know if these instructions work, or if you can direct me on how to request further assistance from the Easy Build community.

Best regards,

Erdi

dsajdak commented 1 year ago

@edasdemirlab We have tried to install this with GCC 11.2.0 and Python 3.9.6 and according to the log files everything seems like it works until it gets to the sanity check and fails:

== 2023-05-09 13:41:42,358 run.py:702 INFO parse_log_for_error (some may be harmless) regExp (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w) found:
gurobi_cl: error while loading shared libraries: libgurobi100.so: cannot open shared object file: No such file or directory
== 2023-05-09 13:41:42,358 run.py:660 WARNING Found 1 errors in command output (output: gurobi_cl: error while loading shared libraries: libgurobi100.so: cannot open shared object file: No such file or directory)
== 2023-05-09 13:41:42,358 easyblock.py:3554 WARNING Sanity check: sanity check command gurobi_cl --help exited with code 127 (output: gurobi_cl: error while loading shared libraries: libgurobi100.so: cannot open shared object file: No such file or directory
)
== 2023-05-09 13:41:42,363 run.py:217 DEBUG run_cmd: running cmd test -f $GRB_LICENSE_FILE (in /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcc/11.2.0/gurobi/10.0.1)
== 2023-05-09 13:41:42,363 run.py:236 INFO running cmd: test -f $GRB_LICENSE_FILE
== 2023-05-09 13:41:42,366 run.py:650 DEBUG cmd "test -f $GRB_LICENSE_FILE" exited with exit code 0 and output:

== 2023-05-09 13:41:42,366 run.py:684 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2023-05-09 13:41:42,366 easyblock.py:3556 INFO sanity check command test -f $GRB_LICENSE_FILE ran successfully! (output: )
== 2023-05-09 13:41:42,371 run.py:217 DEBUG run_cmd: running cmd python -c 'import gurobipy' (in /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcc/11.2.0/gurobi/10.0.1)
== 2023-05-09 13:41:42,371 run.py:236 INFO running cmd: python -c 'import gurobipy'
== 2023-05-09 13:41:42,431 run.py:684 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2023-05-09 13:41:42,431 easyblock.py:3554 WARNING Sanity check: sanity check command python -c 'import gurobipy' exited with code 1 (output: Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcc/11.2.0/gurobi/10.0.1/lib/python3.9/site-packages/gurobipy/__init__.py", line 1, in <module>
    from .gurobipy import *
ImportError: libgurobi100.so: cannot open shared object file: No such file or directory

I'm not seeing anything online to go on so I was hoping to add to your request with EasyBuild team. Is that on a public repo that I could just comment on? Or I can share the full log file with you to share with them.

edasdemirlab commented 1 year ago

Hi Dori, thank you for dealing with this!

I believe that they will see and reply if you comment on PR: https://github.com/easybuilders/easybuild-easyconfigs/pull/17847. I will also let them now after you comment.

Alternatively, we can communicate with them on their slack server. I am in touch with their community on slack: https://easybuild.slack.com/ If you join them, you can search "gurobi" in main channel to find our thread.

And of course I can be a bridge between you and them if you prefer. In this case you may send me the log file.

Erdi

@edasdemirlab We have tried to install this with GCC 11.2.0 and Python 3.9.6 and according to the log files everything seems like it works until it gets to the sanity check and fails:

== 2023-05-09 13:41:42,358 run.py:702 INFO parse_log_for_error (some may be harmless) regExp (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w) found:
gurobi_cl: error while loading shared libraries: libgurobi100.so: cannot open shared object file: No such file or directory
== 2023-05-09 13:41:42,358 run.py:660 WARNING Found 1 errors in command output (output: gurobi_cl: error while loading shared libraries: libgurobi100.so: cannot open shared object file: No such file or directory)
== 2023-05-09 13:41:42,358 easyblock.py:3554 WARNING Sanity check: sanity check command gurobi_cl --help exited with code 127 (output: gurobi_cl: error while loading shared libraries: libgurobi100.so: cannot open shared object file: No such file or directory
)
== 2023-05-09 13:41:42,363 run.py:217 DEBUG run_cmd: running cmd test -f $GRB_LICENSE_FILE (in /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcc/11.2.0/gurobi/10.0.1)
== 2023-05-09 13:41:42,363 run.py:236 INFO running cmd: test -f $GRB_LICENSE_FILE
== 2023-05-09 13:41:42,366 run.py:650 DEBUG cmd "test -f $GRB_LICENSE_FILE" exited with exit code 0 and output:

== 2023-05-09 13:41:42,366 run.py:684 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2023-05-09 13:41:42,366 easyblock.py:3556 INFO sanity check command test -f $GRB_LICENSE_FILE ran successfully! (output: )
== 2023-05-09 13:41:42,371 run.py:217 DEBUG run_cmd: running cmd python -c 'import gurobipy' (in /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcc/11.2.0/gurobi/10.0.1)
== 2023-05-09 13:41:42,371 run.py:236 INFO running cmd: python -c 'import gurobipy'
== 2023-05-09 13:41:42,431 run.py:684 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2023-05-09 13:41:42,431 easyblock.py:3554 WARNING Sanity check: sanity check command python -c 'import gurobipy' exited with code 1 (output: Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcc/11.2.0/gurobi/10.0.1/lib/python3.9/site-packages/gurobipy/__init__.py", line 1, in <module>
    from .gurobipy import *
ImportError: libgurobi100.so: cannot open shared object file: No such file or directory

I'm not seeing anything online to go on so I was hoping to add to your request with EasyBuild team. Is that on a public repo that I could just comment on? Or I can share the full log file with you to share with them.

Hi Dori, thank you for dealing with this!

I believe that they will see and reply if you comment on PR: https://github.com/easybuilders/easybuild-easyconfigs/pull/17847. I will also let them now after you comment.

Alternatively, we can communicate with them on their slack server. I am in touch with their community on slack: https://easybuild.slack.com/ If you join them, you can search "gurobi" in main channel to find our thread.

And of course I can be a bridge between you and them if you prefer. In this case you may send me the log file.

Erdi

aebruno commented 1 year ago

@edasdemirlab I was able to get this to build:

== sanity checking...
  >> file 'bin/grbprobe' found: OK
  >> file 'bin/grbtune' found: OK
  >> file 'bin/gurobi_cl' found: OK
  >> file 'bin/gurobi.sh' found: OK
  >> (non-empty) directory 'matlab' found: OK
  >> loading modules: gcccore/11.2.0, gurobi/10.0.1...
  >> running command 'gurobi_cl --help' ...
  >> result for command 'gurobi_cl --help': OK
  >> running command 'test -f $GRB_LICENSE_FILE' ...
  >> result for command 'test -f $GRB_LICENSE_FILE': OK
  >> running command 'python -c 'import gurobipy'' ...
  >> result for command 'python -c 'import gurobipy'': OK

I also verified that this module works in our test environment:

$ module load gcc
$ module load gurobi
Gurobi shell based on Python 3.9.6 can be launched with command `gurobi.sh`
Gurobi Python Interface can be loaded in Python 3.9.6 with 'import gurobipy'

$ gurobi.sh
Python 3.9.6 (default, Oct 28 2022, 15:17:31)
[GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Set parameter TSPort to value xxxx
Set parameter GURO_PAR_SPECIAL
Set parameter TokenServer to value "xxxx"
Set parameter LogFile to value "gurobi.log"

Gurobi Interactive Shell (linux64), Version 10.0.1
Copyright (c) 2023, Gurobi Optimization, LLC
Type "help()" for help

gurobi>

We will be publishing this in the coming days. Thanks for your patience.

edasdemirlab commented 1 year ago

Hi @aebruno,

Thank you for addressing this!

I just tried "module load gcc" and "module load gurobi," but it only loaded Gurobi Version 9.0.2. Am I missing something, or is the updated version still in progress on your side? Thanks!

aebruno commented 1 year ago

I just tried "module load gcc" and "module load gurobi," but it only loaded Gurobi Version 9.0.2. Am I missing something, or is the updated version still in progress on your side? Thanks!

We have not deployed this yet to our production system. We will update this issue once it's complete. Thanks again for your patience.

edasdemirlab commented 1 year ago

Oh, I see. I apologize for my misunderstanding. Thank you once again.

aebruno commented 1 year ago

@edasdemirlab Gurobi 10.0.1 has been published to our production environment. You should be able to load the module now:

$ module load gcc
$ module load gurobi
edasdemirlab commented 1 year ago

Hi @aebruno . Thank you very much!

I gave it a try, but it loaded Gurobi 9.0.2. What am I missing?

[erdidasd@vortex1:~]$ module load gcc
[erdidasd@vortex1:~]$ module load gurobi
[erdidasd@vortex1:~]$ gurobi.sh
Python 3.7.4 (default, Oct 29 2019, 10:15:53) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Using license file /util/academic/gurobi/licenses/gurobi.lic
Set parameter TokenServer to value srv-m14-36.cbls.ccr.buffalo.edu
Set parameter TSPort to value 46325
Set parameter LogFile to value gurobi.log
Gurobi Interactive Shell (linux64), Version 9.0.2
Copyright (c) 2020, Gurobi Optimization, LLC
Type "help()" for help
gurobi> exit()
aebruno commented 1 year ago

@edasdemirlab please ensure you're using the new software modules. See here. Follow instructions in big yellow box.

edasdemirlab commented 1 year ago

@aebruno Sorry for overlooking this. I haven't been active in CCR for a while. I have now switched to the new module, and it seems to be working. However, I'm super confused with the following two cases. I would appreciate it if you can clarify what is going on.

Allow me to explain.

I tried running my main.py (a) loading gcc and gurobi modules, and (b) without loading them.

(a) produces an error in importing gurobipy, (b) works successfully. Should I not load the modules before using them?

Case (a) produces an error in importing gurobipy:

Last login: Wed May 17 16:49:06 2023 from cld-ondemand.cbls.ccr.buffalo.edu
[erdidasd@vortex2:~]$ module load gcc
[erdidasd@vortex2:~]$ module load gurobi
Gurobi shell based on Python 3.9.6 can be launched with command `gurobi.sh`
Gurobi Python Interface can be loaded in Python 3.9.6 with 'import gurobipy'
[erdidasd@vortex2:~]$ cd 2023-job-scheduling-fire-fighting/
[erdidasd@vortex2:~/2023-job-scheduling-fire-fighting]$ source env/bin/activate
(env) [erdidasd@vortex2:~/2023-job-scheduling-fire-fighting]$ python main.py
Traceback (most recent call last):
  File "/user/erdidasd/2023-job-scheduling-fire-fighting/main.py", line 7, in <module>
    import mip_setup as mip_setup
  File "/user/erdidasd/2023-job-scheduling-fire-fighting/mip_setup.py", line 1, in <module>
    import gurobipy as gp
  File "/cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcccore/11.2.0/gurobi/10.0.1/lib/python3.9/site-packages/gurobipy/__init__.py", line 1, in <module>
    from .gurobipy import *
ImportError: /cvmfs/soft.ccr.buffalo.edu/versions/2023.01/easybuild/software/avx512/Compiler/gcccore/11.2.0/gurobi/10.0.1/lib/python3.9/site-packages/gurobipy/gurobipy.so: undefined symbol: _PyGen_Send
(env) [erdidasd@vortex2:~/2023-job-scheduling-fire-fighting]$

Case (b) works successfully:

Last login: Wed May 17 16:22:40 2023 from cld-ondemand.cbls.ccr.buffalo.edu
[erdidasd@vortex1:~]$ cd 2023-job-scheduling-fire-fighting/
[erdidasd@vortex1:~/2023-job-scheduling-fire-fighting]$ source env/bin/activate
(env) [erdidasd@vortex1:~/2023-job-scheduling-fire-fighting]$ python main.py
Restricted license - for non-production use only - expires 2024-10-28
Set parameter MIPFocus to value 3
Set parameter Presolve to value 2
Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (linux64)
CPU model: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz, instruction set [SSE2|AVX|AVX2|AVX512]
Thread count: 32 physical cores, 32 logical processors, using up to 32 threads
Optimize a model with 535 rows, 351 columns and 2593 nonzeros
Model fingerprint: 0x70b549cc
Variable types: 31 continuous, 320 integer (320 binary)
Coefficient statistics:
  Matrix range     [4e-02, 4e+03]
  Objective range  [1e-06, 1e+00]
  Bounds range     [1e+00, 1e+00]
  RHS range        [1e-06, 1e+04]
Presolve removed 302 rows and 182 columns
Presolve time: 0.08s
Presolved: 233 rows, 169 columns, 1349 nonzeros
Variable types: 11 continuous, 158 integer (158 binary)
Found heuristic solution: objective 1.1489779
Root relaxation presolve removed 6 rows and 0 columns
Root relaxation presolved: 227 rows, 169 columns, 1291 nonzeros
Root relaxation: objective 2.891711e+00, 65 iterations, 0.00 seconds (0.00 work units)
    Nodes    |    Current Node    |     Objective Bounds      |     Work
 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time
     0     0    2.89171    0   21    1.14898    2.89171   152%     -    0s
H    0     0                       1.2261830    2.89171   136%     -    0s
H    0     0                       1.2625757    2.89171   129%     -    0s
H    0     0                       1.3970957    2.89171   107%     -    0s
H    0     0                       1.4599481    2.81581  92.9%     -    0s
H    0     0                       1.4645961    2.81581  92.3%     -    0s
H    0     0                       1.5146632    2.73973  80.9%     -    0s
     0     0    2.66555    0   63    1.51466    2.66555  76.0%     -    0s
H    0     0                       1.6074089    2.66555  65.8%     -    0s
H    0     0                       1.6245783    2.66555  64.1%     -    0s
H    0     0                       1.6399907    2.66555  62.5%     -    0s
     0     0    2.47978    0   60    1.63999    2.47978  51.2%     -    0s
H    0     0                       1.7127184    2.47978  44.8%     -    0s
H    0     0                       1.7700353    2.46593  39.3%     -    0s
     0     0    1.77004    0   58    1.77004    1.77004  0.00%     -    0s

Cutting planes:
  Learned: 8
  Gomory: 81
  Lift-and-project: 34
  Cover: 19
  Implied bound: 36
  Projected implied bound: 3
  Clique: 29
  MIR: 32
  StrongCG: 7
  Flow cover: 57
  GUB cover: 2
  Zero half: 98
  RLT: 7
  Relax-and-lift: 37

Explored 1 nodes (524 simplex iterations) in 0.36 seconds (0.13 work units)
Thread count was 32 (of 32 available processors)

Solution count 10: 1.77004 1.71272 1.63999 ... 1.26258

Optimal solution found (tolerance 1.00e-04)
Best objective 1.770035344394e+00, best bound 1.770035344394e+00, gap 0.0000%
(env) [erdidasd@vortex1:~/2023-job-scheduling-fire-fighting]$ 
aebruno commented 1 year ago

If you're having issues please submit a ccr help request. These GithHub issues are only for build requests. Thanks!

edasdemirlab commented 1 year ago

Alright, thanks! I thought that this issue might be related to the software build, so I apologize for asking about it. I have now submitted a new CCR help ticket. I appreciate your assistance throughout the process! Best regards.

aebruno commented 1 year ago

Alright, thanks! I thought that this issue might be related to the software build, so I apologize for asking about it. I have now submitted a new CCR help ticket. I appreciate your assistance throughout the process! Best regards.

Thank you! and no worries at all.