Idea: Julia DEPOT in bundle

fonsp commented 1 year ago

Instead of generating a sysimage (250MB) we could also create a Julia depot dedicated to launching Pluto, prepopulate it during our build phase, and include it as our assets.

This will also really improve the app's stability: people can safely mess with their own .julia folder and the Pluto app will still work.

We need to make sure that the Pluto notebooks still use the original DEPOT_PATH setting.

➜  ~ cd Downloads 
➜  Downloads mkdir test_julia_depot
➜  Downloads cd test_julia_depot 
➜  test_julia_depot JULIA_DEPOT_PATH="$(pwd)/hello" julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.3 (2022-11-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> DEPOT_PATH
1-element Vector{String}:
 "/Users/fons/Downloads/test_julia_depot/hello"

(@v1.8) pkg> st
  Installing known registries into `~/Downloads/test_julia_depot/hello`
Status `~/Downloads/test_julia_depot/hello/environments/v1.8/Project.toml` (empty project)

(@v1.8) pkg> add Pluto
    Updating registry at `~/Downloads/test_julia_depot/hello/registries/General.toml`
   Resolving package versions...
   Installed Tricks ────────────────────── v0.1.6
   Installed RegistryInstances ─────────── v0.1.0
   Installed Scratch ───────────────────── v1.1.1
   Installed TableTraits ───────────────── v1.0.1
   Installed OpenSSL_jll ───────────────── v1.1.19+0
   Installed BitFlags ──────────────────── v0.1.7
   Installed LoggingExtras ─────────────── v1.0.0
   Installed Preferences ───────────────── v1.3.0
   Installed SimpleBufferStream ────────── v1.1.0
   Installed DataAPI ───────────────────── v1.14.0
   Installed RelocatableFolders ────────── v1.0.0
   Installed Tables ────────────────────── v1.10.0
   Installed IteratorInterfaceExtensions ─ v1.0.0
   Installed PrecompileSignatures ──────── v3.0.3
   Installed ExproniconLite ────────────── v0.7.11
   Installed JLLWrappers ───────────────── v1.4.1
   Installed IniFile ───────────────────── v0.5.1
   Installed URIs ──────────────────────── v1.4.1
   Installed DataValueInterfaces ───────── v1.0.0
   Installed MsgPack ───────────────────── v1.1.0
   Installed OrderedCollections ────────── v1.4.1
   Installed TranscodingStreams ────────── v0.9.11
   Installed FuzzyCompletions ──────────── v0.5.1
   Installed MIMEs ─────────────────────── v0.1.4
   Installed OpenSSL ───────────────────── v1.3.3
   Installed HTTP ──────────────────────── v1.7.1
   Installed Configurations ────────────── v0.17.4
   Installed MbedTLS ───────────────────── v1.1.7
   Installed CodecZlib ─────────────────── v0.7.0
   Installed HypertextLiteral ──────────── v0.9.4
   Installed LazilyInitializedFields ───── v1.2.0
   Installed Pluto ─────────────────────── v0.19.19
  Downloaded artifact: OpenSSL
    Updating `~/Downloads/test_julia_depot/hello/environments/v1.8/Project.toml`
  [c3e4b0f8] + Pluto v0.19.19
    Updating `~/Downloads/test_julia_depot/hello/environments/v1.8/Manifest.toml`
  [d1d4a3ce] + BitFlags v0.1.7
  [944b1d66] + CodecZlib v0.7.0
  [5218b696] + Configurations v0.17.4
  [9a962f9c] + DataAPI v1.14.0
  [e2d170a0] + DataValueInterfaces v1.0.0
⌅ [55351af7] + ExproniconLite v0.7.11
  [fb4132e2] + FuzzyCompletions v0.5.1
  [cd3eb016] + HTTP v1.7.1
  [ac1192a8] + HypertextLiteral v0.9.4
  [83e8ac13] + IniFile v0.5.1
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.4.1
  [0e77f7df] + LazilyInitializedFields v1.2.0
  [e6f89c97] + LoggingExtras v1.0.0
  [6c6e2e6c] + MIMEs v0.1.4
  [739be429] + MbedTLS v1.1.7
  [99f44e22] + MsgPack v1.1.0
  [4d8831e6] + OpenSSL v1.3.3
  [bac558e1] + OrderedCollections v1.4.1
  [c3e4b0f8] + Pluto v0.19.19
  [91cefc8d] + PrecompileSignatures v3.0.3
  [21216c6a] + Preferences v1.3.0
  [2792f1a3] + RegistryInstances v0.1.0
  [05181044] + RelocatableFolders v1.0.0
  [6c6a2e73] + Scratch v1.1.1
  [777ac1f9] + SimpleBufferStream v1.1.0
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.10.0
  [3bb67fe8] + TranscodingStreams v0.9.11
  [410a4b4d] + Tricks v0.1.6
  [5c2747f8] + URIs v1.4.1
  [458c3c95] + OpenSSL_jll v1.1.19+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.1
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [83775a58] + Zlib_jll v1.2.12+3
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  38 dependencies successfully precompiled in 21 seconds

(@v1.8) pkg> 
➜  test_julia_depot JULIA_DEPOT_PATH="$(pwd)/hello" julia -e "import Pluto; Pluto.run()"
[ Info: Loading...
[ Info: Listening on: 127.0.0.1:1234, thread id: 1
┌ Info: 
└ Opening http://localhost:1234/?secret=TBzlMdEc in your default browser... ~ have fun!
┌ Info: 
│ Press Ctrl+C in this terminal to stop Pluto
└ 
^C

┌ Info: 
│ Closing Pluto... Restart Julia for a fresh session. 
│ 
│ Have a nice day! 🎈
└ 
[ Info: Server on 127.0.0.1:1234 closing
➜  test_julia_depot

This depot is about 100MB, split down into these subfolders:

(the compiled/Pluto folder is 56MB)

We can safely delete the registries folder (I tested), which will save some storage and a large number of files on Windows. Julia will not download the registries on launch.

You cannot delete packages.

You can delete compiled (70MB), but this will mean a slow launch.

With registries deleted, the depot is 94MB, or 41MB after compression.

ctrekker commented 1 year ago

On windows this creates issues when users try to open the app, since Julia will try to write to the depot directory. When installed the depot directory is located in C:\Program Files, which is readonly. Perhaps we move the depot into AppData or another writable directory.

fonsp commented 1 year ago

ah shitty. But electron has API to achieve this sort of thing, right?

ctrekker commented 1 year ago

Yeah I created #69 to hopefully fix this issue by copying the packaged depot to AppData, it takes an extra second or two the first run but after that everything works the same.

JuliaPluto / PlutoDesktop

Idea: Julia DEPOT in bundle #56