search

COPT-Public / COPT.jl

Julia interface for COPT (Cardinal Optimizer)
Other
36 stars 4 forks source link

Automated binary installs #3

Closed odow closed 2 years ago

odow commented 2 years ago

Leaving breadcrumbs to how we automatically install the Gurobi binary on linux (users still need to create and grbgetkey a license):

Create an artifacts file https://github.com/jump-dev/Gurobi.jl/blob/master/Artifacts.toml

Here's the script to create the hashes https://github.com/jump-dev/Gurobi.jl/pull/437#issue-1057702916

Don't write a deps file if not needed https://github.com/jump-dev/Gurobi.jl/blob/e1a163acfa8aca98059bbef43c0aefe0f540591d/deps/build.jl#L172-L175

Load the artifact https://github.com/jump-dev/Gurobi.jl/blob/e1a163acfa8aca98059bbef43c0aefe0f540591d/src/Gurobi.jl#L16-L23

lschork2 commented 2 years ago

Thank you for the detailed instructions. I have added this for Linux and Mac. Two questions:

(1) Do I understand correctly that only tar files can be extracted, but no zip files? So if we wanted to do it for Windows, we would need to release the COPT package as tar.gz also on Windows?

(2) When we update the Artifacts.toml to a new COPT version and users of the old version run Pkg.update("COPT"), would this download the new version?

odow commented 2 years ago

1) Yeah, it has to be a .tar.gz file, even on Windows :( 2) We need to create tagged releases, which I'm about to reply to Qi about/open a new issue. But yes, if you update Artifacts, then tag a new version, when people run update it will download the new version. It's all very smooth!

odow commented 2 years ago

The binaries are working great on linux and Mac: https://github.com/COPT-Public/COPT.jl/pull/6

If you can add binaries for Windows, then installing will be a breeze, and we can remove all the hacky stuff in /deps.

COPT-Public commented 2 years ago

We made a tar.gz package for Windows and adjust the code. When we test it, the installation stuck at loading the dll. Full error messages are:

Precompiling project...
  ✗ COPT
  0 dependencies successfully precompiled in 1 seconds (38 already precompiled)
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

julia> Pkg.precompile()
Precompiling project...
  ✗ COPT
  0 dependencies successfully precompiled in 1 seconds (38 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f]

Failed to precompile COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f] to C:\Users\Huangfu\.julia\compiled\v1.7\COPT\jl_22F.tmp.
ERROR: LoadError: could not load library "C:\Users\Huangfu\.julia\artifacts\ac107ec0930ed47ecbe87665f0aeed39be40c904\copt40\bin\copt.dll"
Access is denied.

I tested by manually clicking to grant it access right, and then it works. I repeated by overwriting the dll file by the one I extracted from the tar.gz, and it also works.

Do you have any experience regarding fixing this issue? @odow

odow commented 2 years ago

What are the permissions on the dll before you tar it? And permissions on the tar? Julia doesn't do anything special, it just extracts the tar, keeping whatever permissions were there in the first place.

I don't have a windows machine to test, unfortunately.

On Fri, 29 Apr 2022, 9:11 PM COPT-Public, @.***> wrote:

We made a tar.gz package for Windows and adjust the code. When we test it, the installation stuck at loading the dll. Full error messages are:

Precompiling project...

✗ COPT

0 dependencies successfully precompiled in 1 seconds (38 already precompiled)

1 dependency errored. To see a full report either run import Pkg; Pkg.precompile() or load the package

julia> Pkg.precompile()

Precompiling project...

✗ COPT

0 dependencies successfully precompiled in 1 seconds (38 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f]

Failed to precompile COPT [227a2e2d-e949-4d8e-a1da-7384fe6f0b9f] to C:\Users\Huangfu.julia\compiled\v1.7\COPT\jl_22F.tmp.

ERROR: LoadError: could not load library "C:\Users\Huangfu.julia\artifacts\ac107ec0930ed47ecbe87665f0aeed39be40c904\copt40\bin\copt.dll"

Access is denied.

I tested by manually clicking to grant it access right, and then it works. I repeated by overwriting the dll file by the one I extracted from the tar.gz, and it also works.

Do you have any experience regarding fixing this issue? @odow https://github.com/odow

— Reply to this email directly, view it on GitHub https://github.com/COPT-Public/COPT.jl/issues/3#issuecomment-1113087417, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6MQJP34JLGRUPLQXW6DXTVHORUDANCNFSM5UN7SIUQ . You are receiving this because you were mentioned.Message ID: @.***>

COPT-Public commented 2 years ago

It was created using the tar tool provided by Windows. If unzip it on Windows manually, things work fine. I am afraid that it may take much long to get it fixed as it seems no one has direct experiences regarding this new issue. Is there any other solver who provides automatic download for Windows that we can learn from?

odow commented 2 years ago

Here's the HiGHS binary for Windows:

https://github.com/JuliaBinaryWrappers/HiGHS_jll.jl/releases/download/HiGHS-v1.2.2%2B0/HiGHS.v1.2.2.x86_64-w64-mingw32-cxx11.tar.gz

https://github.com/JuliaBinaryWrappers/HiGHS_jll.jl/releases/tag/HiGHS-v1.2.2%2B0

If you're still having trouble, email me the tarball, and I'll take a look.

COPT-Public commented 2 years ago

I am a bit puzzled by the workflow of HiGHS regarding the Windows automatic build. The tarball of our Windows version is online at https://pub.shanshu.ai/download/copt/4.0.7/win64/CardinalOptimizer-4.0.7-win64.tar.gz

COPT-Public commented 2 years ago

Interesting that, on Windows, the extracted .exe files have "executable" access, but the .dll files don't.

lschork2 commented 2 years ago

We now use scripts/generate_tar_package.jl to convert the COPT zip package to tar.gz. When the script is used on Windows, then the .dll file obtained by extracting the tar.gz has executable permission and the automatic installation works. When the script is used on Linux, then the .dll file does not have executable permission and the automatic installation fails.

So for now we build the package on Windows.