JuliaLang / PackageCompiler.jl

Compile your Julia Package
https://julialang.github.io/PackageCompiler.jl/dev/
MIT License
1.4k stars 186 forks source link

Bundle dynamically selected libstdc++ #853

Closed sloede closed 9 months ago

sloede commented 10 months ago

Julia by default uses libstdcxxprobe() on Linux systems to figure out whether to load the system libstdc++ or the one bundled with Julia. Not doing this can cause problems if other libraries used by a Julia process are compiled against a system libstdc++ that is newer than the one provided by Julia

PackageCompiler currently always bundles the library shipped with Julia, not honoring the dynamic selection. This PR aims to remedy this by searching for the libstdc++ that will be bundled in the directory where the currently loaded libstdc++ resides.

xref: https://discourse.julialang.org/t/precedence-of-local-and-julia-shipped-shared-libraries/104258?u=sloede

@staticfloat it would be great if you could have a look if this seems like a reasonable implementation to you.

@giordano thanks a lot for the initial pointer on how get the libstdc++ actually used by Julia.

codecov[bot] commented 10 months ago

Codecov Report

Merging #853 (968751a) into master (a83df58) will increase coverage by 0.78%. Report is 3 commits behind head on master. The diff coverage is 97.91%.

@@            Coverage Diff             @@
##           master     #853      +/-   ##
==========================================
+ Coverage   84.15%   84.94%   +0.78%     
==========================================
  Files           3        3              
  Lines         751      797      +46     
==========================================
+ Hits          632      677      +45     
- Misses        119      120       +1     
Files Coverage Δ
src/PackageCompiler.jl 93.44% <97.91%> (+0.34%) :arrow_up:

... and 1 file with indirect coverage changes

sloede commented 10 months ago

@staticfloat it would be great to get another review from you, otherwise I'll go ahead and merge this within the next few days (unless @KristofferC objects).

sloede commented 10 months ago

@bgeihe could you please try once more if this PR works on your previously broken system, now that I simplified the library alias code? If it does (and the tests pass), we can merge this PR.

benegee commented 10 months ago

Confirmed!

PallHaraldsson commented 9 months ago

It good that you fixed this, but I see what we bundle has grow from 1378688 libstdc++.so.6.0.25, to 18006768 julia-1.8.1/usr/lib/libstdc++.so.6.0.29 to 18763496 libstdc++.so.6.0.30 and is non-small already.

So is 13% larger, and is the 18 MB overhead actually really needed? We're updating to LLVM 16, should likely 17 and 18, and can we expect we need to update newer larger each time?

I was thinking couldn't Julia apps in most cases just dynamically probe and use the system C++ library, at least at some point in the future? I'm not sure what other programs do, C++ based or otherwise.

I believe the C++ library is used by LLVM, not the rest of Julia, and LLVM can optionally be dropped. Would it then be ok to drop that library too?

I realize some JLL (e.g. for Qt) might also use it, but how recent C++ system library can we expect?

In my Linux Mint I found even larger (each) despite same version (each multi-arch or have more optimizations, such as AVX?):

2252096 maí 13 2022 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2295716 maí 13 2022 /usr/lib32/libstdc++.so.6.0.30 18763496 ágú 24 15:20 /snap/julia/86/lib/julia/libstdc++.so.6.0.30