Reduce Workbench size

[sonnyp]

For total size between Workbench 44 and 45 we went from 17MB to 1.5GB (without delta and assuming Sdk is not installed)

The SDK is 800MB Workbench is 500MB and some other stuff

The download size after deltas on my system only ends up being ~ 500MB

That is still a lot to download (and install even if dedup). We want to optimize for "time to paltform", ie how long it takes from clicking "Install" in Software to discovering the cool things you can do with GNOME.

Battle plan:

1. Investigate using Platform instead of Sdk as runtime (needs gcc or clang) (like Workbench 44 did)
2. Make Rust a runtime dependency (either tell user to install the sdk extension or download something else from somewhere)
3. Make Vala a runtime dependency (how much do we shave?)
4. Make docs a runtime dependency (either tell user to install the sdk extension or download something else from somewhere)

[sonnyp]

I would also like to make Workbench easy to contribute to so the download size for contributing to Workbench is also to take into consideration but secondary.

[UrtsiSantsi]

How about making additional languages separate extension flatpaks? It will include runtimes, sdks, demo sources and everything needed. When they are installed Workbench will detect them and add them to the code language menu. Or they can be in the menu, but not installed and trying to use them will ask if the user want them installed. This will save both size and build time. Vala can be another extension, minus the parts that are used by other languages, like Rust.

[Hofer-Julian]

As an additional data point, this is what I get when measuring the size of /app:

[📦 re.sonny.Workbench /]$ du -sh /app/*/ | sort -rh | head
656M    /app/lib/
297M    /app/share/
284M    /app/bin/
32K /app/libexec/
8,0K    /app/etc/

/app/lib gives

[📦 re.sonny.Workbench /]$ ls -hlS /app/lib | head
-rw-r--r--. 5 julian julian 170M  1. Jan 1970  libLLVM-16-rust-1.72.1-stable.so
-rw-r--r--. 3 julian julian 120M  1. Jan 1970  librustc_driver-802a454189e87861.so
-rwxr-xr-x. 3 julian julian  12M  1. Jan 1970  libvala-0.56.so.0.0.0
-rw-r--r--. 5 julian julian  11M  1. Jan 1970  libstd-b850d2b001350814.so
-rwxr-xr-x. 2 julian julian 6,6M  1. Jan 1970  libvte-2.91-gtk4.so.0
-rwxr-xr-x. 3 julian julian 5,8M  1. Jan 1970  libvaladoc-0.56.so.0.0.0
-rwxr-xr-x. 3 julian julian 4,5M  1. Jan 1970  libgee-0.8.so.2.6.1
-rw-r--r--. 5 julian julian 3,1M  1. Jan 1970  libtest-3b56f96151b75935.so
-rwxr-xr-x. 3 julian julian 2,6M  1. Jan 1970  libgvc.so.6.0.0

/app/share/doc/ gives

[📦 re.sonny.Workbench /]$ du -sh /app/share/doc/*/ | sort -rh | head
69M /app/share/doc/gtk4/
30M /app/share/doc/pipewire/
17M /app/share/doc/libadwaita-1/
17M /app/share/doc/gdk4/
12M /app/share/doc/systemd/
11M /app/share/doc/groff-1.22.4/
9,2M    /app/share/doc/gettext/
8,4M    /app/share/doc/libhandy-1/
7,7M    /app/share/doc/gtksourceview5/
7,2M    /app/share/doc/openssl/

/app/bin gives

[📦 re.sonny.Workbench /]$ ls -hlS /app/bin | head
-rwxr-xr-x. 3 julian julian  44M  1. Jan 1970  rust-analyzer
-rwxr-xr-x. 3 julian julian  36M  1. Jan 1970  cargo
-rwxr-xr-x. 3 julian julian  29M  1. Jan 1970  mold
-rwxr-xr-x. 3 julian julian  24M  1. Jan 1970  cargo-capi
-rwxr-xr-x. 3 julian julian  24M  1. Jan 1970  cargo-cinstall
-rwxr-xr-x. 3 julian julian  24M  1. Jan 1970  cargo-cbuild
-rwxr-xr-x. 3 julian julian  19M  1. Jan 1970  uncrustify
-rwxr-xr-x. 3 julian julian  15M  1. Jan 1970  rustdoc
-rwxr-xr-x. 2 julian julian  14M  1. Jan 1970  biome

The obvious things that we can remove already are:

• /app/share/doc/pipewire/
• /app/bin/cargo-capi
• /app/bin/cargo-cinstall
• /app/bin/cargo/cbuild
• /app/bin/rustdoc

[Hofer-Julian]

Investigate using Platform instead of Sdk as runtime (needs gcc or clang) (like Workbench 44 did)

As discussed with @sonnyp, I wonder if that makes things worse. I would expect that many (most?) people using Workbench (including newcomers) will want GNOME Builder installed as well. And Builder uses the Sdk as runtime too. By using the Sdk we share disk space.

Make Vala a runtime dependency (how much do we shave?)

According to the data above, that doesn't seem to save too much space (as @sonnyp already suspected). In case we go for a convenient optional dependency approach for docs, Rust (and future languages), it might still make sense to make Vala optional as well. That way, there would be a minimal Workbench product that can be extended.

Make docs a runtime dependency (either tell user to install the sdk extension or download something else from somewhere)

That makes a lot of sense to me in any way. While they are useful for offline usage, its usefulness is for sure limited by requiring users to convert C docs into their language of choice: GJS, Vala or Rust. I think we should recommend people to use the language specific docs anyway.

[sonnyp]

The obvious things that we can remove

I'm afraid it doesn't move the needle much. I'd rather spend the time on runtime extensions.

I wonder if that makes things worse. I would expect that many (most?) people using Workbench (including newcomers) will want GNOME Builder installed as well. And Builder uses the Sdk as runtime too. By using the Sdk we share disk space.

We are optimizing for initial download time. The chances of the users having org.gnome.Platform installed is closer to 100% than org.gnome.Sdk.

Disk space isn't a concern; Flatpak will deduplicate if the user has the Sdk installed anyway.

In case we go for a convenient optional dependency approach for docs, Rust (and future languages), it might still make sense to make Vala optional as well.

Agree.

I think we should recommend people to use the language specific docs anyway.

Yeah, the current situation is really not great. My next big project might be interactive and multi-lang docs / references.

[Hofer-Julian]

Focusing on runtime extension sounds good to me.

[sonnyp]

@UrtsiSantsi I'm not sure yet if Flatpak extensions is the best mechanism.

I'd prefer Workbench being able to download the dependencies itself rather than tell the user “Please type flatpak install [--user] ... in the termina and restartl”.

I'm not aware of a mechanism that allows an app to install flatpak extensions without punching sandbox holes which I don't want to do.

[sonnyp]

https://github.com/flatpak/flatpak/issues/3317 confirms that it's not currently possible.

However, it is desired, so perhaps it's best to go with Flatpak extensions, tell user to type something in the terminal and restart Workbench. Until the feature is implemented.

[sonnyp]

The easiest way forward is:

1. Stay on org.gnome.Sdk
2. Request user to flatpak install extension-id for Vala / Rust / Docs and restart Workbench

I think we can design a nice dialog that explains things nicely. Maybe @bragefuglseth can make one of his fantastic illustration for it? :D

---

"Extensions"

Install offline API references (hidden or :heavy_check_mark: if already installed)

flatpak install  flathub org.gnome.Sdk.Docs//45

org.freedesktop.Sdk.Extension.rust-stable//23.08 org.freedesktop.Sdk.Extension.vala//23.08

Install Rust support (hidden or :heavy_check_mark: if already installed)

flatpak install org.freedesktop.Sdk.Extension.rust-stable//23.08

Install Vala support (hidden or :heavy_check_mark: if already installed)

flatpak install org.freedesktop.Sdk.Extension.vala//23.08

Please restart Workbench (if Extensions was open, it should open on restart)

[sonnyp]

The dialog should only show when one of these is needed:

• Users attempts to "Run" Rust
• Users attempts to "Run" Vala
• Users opens "Manuals"

We could also show "Extensions" entry in Workbench menu to show the dialog.

[Hofer-Julian]

Not saying that this is a blocker, but let's keep in mind that we loose a fair bit of control that way. The rust sdk for example always gets the newest Rust version, even within the same sdk version. That means that we can't ship Cargo.lock anymore IIRC.

We can still specify version ranges with Cargo.toml, so breakage shouldn't occur under normal circumstances.

[sonnyp]

That's good to know.

I did try to add my own extension to Cargo.toml and then "Run" and it didn't work because of the Cargo.lock.

Do we want/need the Cargo.lock anyway?

[Hofer-Julian]

No, I liked adding Cargo.lock, because that way users have exactly the same dependencies as we do.

But if we can't control the Rust version it's not worth it IMO

We also need to remove --locked from the cargo call then.

[sonnyp]

Ha ok to solve the problem of letting users add dependencies we could have just dropped --locked then?

[sonnyp]

The rust sdk for example always gets the newest Rust version, even within the same sdk version. That means that we can't ship Cargo.lock anymore IIRC.

This is a bit sad indeed. Any chance the situation improves as Rust stabilizes? Node.js freedesktop extensions for example are pinned to major releases.

[Hofer-Julian]

They could always add dependencies

They had to do it with cargo add or run cargo update afterwards

[Hofer-Julian]

The rust sdk for example always gets the newest Rust version, even within the same sdk version. That means that we can't ship Cargo.lock anymore IIRC.

This is a bit sad indeed. Any chance the situation improves as Rust stabilizes? Node.js freedesktop extensions for example are pinned to major releases.

This isn't really a matter of Rust stabilizing. It just follows the Rust release schedule which happens to be faster than freedesktop sdk schedule. Not sure if people would like it if they couldn't use the newest Rust version when using the sdk.

[sonnyp]

In the case of Node.js there is a Node.js 18 and Node.js 20 extensions for both Freedesktop 22.08 and 23.08

[sonnyp]

Here is a first iteration I'm going to start with

[bragefuglseth]

Maybe @bragefuglseth can make one of his fantastic illustration for it? :D

Ping me when the rest of the dialog has been settled on, so I know what to illustrate for 👍

[Hofer-Julian]

Here is a first iteration I'm going to start with

Will that work no matter if the user installed Workbench user or system wide?

[sonnyp]

Added as a todo to https://github.com/sonnyp/Workbench/pull/639

[sonnyp]

flatpak install installs in the prefix in which you have the specified remote installed, in this case flathub.

I think it's safe to proceed / assume.

[Hofer-Julian]

flatpak install installs in the prefix in which you have the specified remote installed, in this case flathub.

I think it's safe to proceed / assume.

We should make sure to guide the user if they have both remotes installed though :)

[UrtsiSantsi]

Trying to build 6c2f927 with Builder I get "error: Requested extension org.freedesktop.Sdk.Extension.llvm16/x86_64/45 not installed".