JuliaLang / PackageCompiler.jl

Compile your Julia Package
https://julialang.github.io/PackageCompiler.jl/dev/
MIT License
1.42k stars 191 forks source link

from scratch: running in a static environment #479

Open maleadt opened 3 years ago

maleadt commented 3 years ago

I'm trying to run PackageCompiler-generated binaries in a very bare environment -- a Docker image that starts with only from scratch -- and I run into a few issues before I could execute the generated app. Documenting those steps here in case we want to add a static mode (where additional libraries are bundled):

  1. create_app("MyApp", "MyAppCompiled")
  2. Figure out which libraries our generated binary uses, and copy those over to the lib folder:
    ~/PackageCompiler/examples$ ldd MyAppCompiled/bin/MyApp
            linux-vdso.so.1 (0x00007ffd86380000)
            libc.so.6 => /usr/lib/libc.so.6 (0x00007f30de3b4000)
            libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f30daa10000)
            librt.so.1 => /usr/lib/librt.so.1 (0x00007f30daa05000)
            libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f30da9e3000)
            libm.so.6 => /usr/lib/libm.so.6 (0x00007f30da51f000)
            /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f30e4624000)
    ~/PackageCompiler/examples$ cp /lib64/ld-linux-x86-64.so.2 /usr/lib/libc.so.6 MyAppCompiled/lib
    # etc
  3. Move MyApp.so from the bin to lib folder, or Julia doesn't seem able to pick it up
  4. Build a container:
    FROM scratch
    COPY MyAppCompiled /
    ENV LD_LIBRARY_PATH=/lib
    ENTRYPOINT ["/lib/ld-linux-x86-64.so.2"]
    CMD ["/bin/MyApp"]
    ~/PackageCompiler/examples$ docker build . -t MyAppCompiled && docker run --rm -it MyAppCompiled
    Sending build context to Docker daemon  268.1MB
    Step 1/5 : FROM scratch
    ---> 
    Step 2/5 : COPY MyAppCompiled /
    ---> Using cache
    ---> f12c51bac908
    Step 3/5 : ENV LD_LIBRARY_PATH=/lib
    ---> Using cache
    ---> 44b8d2554a8f
    Step 4/5 : ENTRYPOINT ["/lib/ld-linux-x86-64.so.2"]
    ---> Running in 1fca95357268
    Removing intermediate container 1fca95357268
    ---> 04e51f9e29fa
    Step 5/5 : CMD ["/bin/MyApp"]
    ---> Running in 0f57d7947861
    Removing intermediate container 0f57d7947861
    ---> 83254aa9ba6a
    Successfully built 83254aa9ba6a
    Successfully tagged pkgcompiler:latest
    ARGS = String[]
    Base.PROGRAM_FILE = "/bin/MyApp"
    DEPOT_PATH = ["//"]
    LOAD_PATH = ["@"]
    pwd() = "/"
    Base.active_project() = nothing
    Threads.nthreads() = 1
    Sys.BINDIR = "/lib"
    Dict{Base.PkgId,Module} with 35 entries:
    CRC32c [8bf52ea8-c179-5cab-976a-9e18b702a9bc] => CRC32c
    Printf [de0858da-6303-5e67-8744-51eddeeeb8d7] => Printf
    LinearAlgebra [37e2e46d-f89d-539d-b4ee-838fcccc9c8e] => LinearAlgebra
    UUIDs [cf7118a7-6976-5b1a-9a39-7adc72f591a4] => UUIDs
    REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] => REPL
    Libdl [8f399da3-3557-5675-b5ff-fb832c97cbdb] => Libdl
    Mmap [a63ad114-7e13-5084-954f-fe012c677804] => Mmap
    Example [7876af07-990d-54b4-ab0e-23690620f79a] => Example
    Markdown [d6f4376e-aef5-505a-96c1-9c027394607a] => Markdown
    Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] => Base64
    Random [9a3f8284-a2c9-5f02-9a11-845980a1fd5c] => Random
    Base [top-level] => Base
    Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79] => Profile
    HelloWorldC_jll [dca1746e-5efc-54fc-8249-22745bc95a49] => HelloWorldC_jll
    Unicode [4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5] => Unicode
    Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] => Pkg
    InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240] => InteractiveUtils
    SharedArrays [1a1011a3-84de-559e-8e89-a11a2f7dc383] => SharedArrays
    Serialization [9e88b42a-f829-5b0c-bbe9-9e923198166b] => Serialization
    MyApp [f943f3d7-887a-4ed5-b0c0-a1d6899aa8f5] => MyApp
    Future [9fa8497b-333b-5362-9e8d-4d0656e87820] => Future
    DelimitedFiles [8bb1440f-4735-579b-a4ab-409b98df4dab] => DelimitedFiles
    SparseArrays [2f01184e-e22b-5df5-ae63-d93ebab69eaf] => SparseArrays
    Dates [ade2ca70-3891-5945-98fb-dc099432e06a] => Dates
    Sockets [6462fe0b-24de-5631-8697-dd941f90decc] => Sockets
    Statistics [10745b16-79ce-11e8-11f9-7d13ad32a3b2] => Statistics
    Core [top-level] => Core
    Main [top-level] => Main
    Distributed [8ba89e20-285c-5b6f-9357-94700520ee1b] => Distributed
    Test [8dfed614-e22c-5e08-85e1-65c5234f0b40] => Test
    Logging [56ddb016-857b-54e1-b83d-db4d58db5568] => Logging
    SHA [ea8e919c-243c-51af-8825-aaa63cd721ce] => SHA
    SuiteSparse [4607b0f0-06f3-5cda-b6b1-a6196a1729e9] => SuiteSparse
    FileWatching [7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee] => FileWatching
    LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] => LibGit2

In this funky set-up Julia isn't able to spawn binaries (probably because it then doesn't use the dynamic linker -- unsure how that's set up by default, I haven't dug that deep):

$ docker run --rm -it MyAppCompiled
...
Running a jll package:
HelloWorld artifact at /artifacts/4da0ccfb65e328aad3d3d7f9b5c06a4564cbc53d/bin/hello_world
ERROR: IOError: could not spawn `/artifacts/4da0ccfb65e328aad3d3d7f9b5c06a4564cbc53d/bin/hello_world`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at ./process.jl:99
 [2] #585 at ./process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at ./process.jl:196
 [4] _spawn at ./process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at ./process.jl:439
 [6] run at ./process.jl:438 [inlined]
 [7] (::MyApp.var"#1#2")(::String) at /home/tim/PackageCompiler/examples/MyApp/src/MyApp.jl:34
 [8] (::HelloWorldC_jll.var"#8#9"{MyApp.var"#1#2"})() at /home/tim/.julia/packages/HelloWorldC_jll/QWPAv/src/wrappers/x86_64-linux-gnu.jl:34
 [9] withenv(::HelloWorldC_jll.var"#8#9"{MyApp.var"#1#2"}, ::Pair{String,String}, ::Vararg{Pair{String,String},N} where N) at ./env.jl:161
 [10] hello_world(::MyApp.var"#1#2"; adjust_PATH::Bool, adjust_LIBPATH::Bool) at /home/tim/.julia/packages/HelloWorldC_jll/QWPAv/src/wrappers/x86_64-linux-gnu.jl:33
 [11] hello_world at /home/tim/.julia/packages/HelloWorldC_jll/QWPAv/src/wrappers/x86_64-linux-gnu.jl:17 [inlined]
 [12] real_main() at /home/tim/PackageCompiler/examples/MyApp/src/MyApp.jl:32
 [13] julia_main() at /home/tim/PackageCompiler/examples/MyApp/src/MyApp.jl:11
 [14] julia_main() at ./none:36

$ docker run --rm -it pkgcompiler /artifacts/4da0ccfb65e328aad3d3d7f9b5c06a4564cbc53d/bin/hello_world
Hello, World!

For debuging, you can launch docker with --env LD_DEBUG=libs since we're using the dynamic linker here.

KristofferC commented 3 years ago

Figure out which libraries our generated binary uses, and copy those over to the lib folder:

Do you think it makes sense to try to statically link those or is bundling better?

Move MyApp.so from the bin to lib folder, or Julia doesn't seem able to pick it up

That's strange. It should be set here:

https://github.com/JuliaLang/PackageCompiler.jl/blob/2d8010d8be72d65269d114b037be35081ec7f303/src/embedding_wrapper.c#L48

and just be the name of the app. From CI, it seems to find it next to the executable.

maleadt commented 3 years ago

Do you think it makes sense to try to statically link those or is bundling better?

Since our Yggdrasil-built artifacts are all dynamically linked, we need to bundle those libraries anyway.