Add a "resource bundle" for files like images, fonts, and localization

[futurepaul]

This came up in the #353 discussion. I'll let @cmyr explain it:

My long-term vision for this is that you shouldn't be loading data directly from the file system; there should be a 'resource bundle' that druid exposes that contains various types of resources (image files, font files, localization files) and that is validated at compile time, so that at runtime getting a resource looks something like,

let my_svg: SvgData = env.get_bundle_resource(MY_SVG_IDENTIFIER);

or maybe even in a very fancy world there's a build phase that generates per-app marker 'keys' (like env::Key) for each of the resources in this app's bundle?

[raphlinus]

I do agree that we should have something like this. Here are a few more thoughts.

This intersects deeply with platform mechanisms for bundling resources. On mac, while we're currently generating a single binary as the result of "cargo build" (and I am very reluctant to break this use case), the real way to ship applications is through an app bundle.

Similarly, on Windows it's possible to bundle resources into the .exe file, and there are ways to get at that. I'm not sure if it's still recommended to bundle all resources into the exe, and certainly bigger apps are split into multiple files (and there's a whole nother mechanism for distributing that, either msi or something else).

In both cases, the app bundle has a lot of other important stuff (icons, manifest, digital signatures). So there's a ton of complexity here.

I'm less familiar with the situation on Linux. It seems like there are a lot of choices; it's certainly possible to design a mechanism that feeds reasonably cleanly into traditional distribution packagers such as apt and rpm. But possibly it's better to target flatpak and/or snap. I'd certainly want input from people in the Linux community on this.

As I said above, I don't want to break the "cargo run" experience, so building such a bundle should be opt-in, I think. Ideally we'd set this up so that it's still possible to "cargo run" for development purposes, and it would do a reasonably good job of finding the resources.

[futurepaul]

electron-packager might be a good overview of the possible app bundle outputs

[mendelt]

You can bundle binary data inside executables on all three platforms. I remember seeing support for this in some older languages but I dont think Rust supports this. Maybe through some obscure linker magic? I think the normal way to do this in modern applications is to bundle data in an installer (windows msi) or package file. This just means when installing the files get extracted and put in a predictable place on the file system. I don't have any experience using apple systems but I understand that programs there are just distributed as compressed archives that also unpack into separate files. Wasm applications might be different. They will have their resources hosted on a server somewhere.

[ratmice]

@mendelt rust does have ways to do that, there are the include_str! and include_bytes! macros

[Ralith]

That said, idiomatic applications generally don't bake everything into the executable, and on macOS and Linux there are strong reasons to have external files, e.g. so the OS can find your icon.

[cmyr]

yea, macOS has a well described bundle format, and we should be building this when we build an apple bundle on mac, which I want to be as easy as cargo druid bundle.

[mendelt]

@ratmice Thank you. Learned something today. @cmyr I can imagine druid resources are not the only use-case for bundling stuff with your app. It might also make sense to just have a more generic cargo bundle as a separate tool that can bundle anything and then have druid just access the files from the file system when the bundle is installed.

[cmyr]

@mendelt I think of cargo druid bundle as being a thing that not just bundles up resources but which actually creates a native application package for the given platform.

[mendelt]

@cmyr An application packager like that sounds like a very useful thing to have. I just wonder how much of it should be specific to Druid. But you can always start by building something that works great for Druid and then extract out the generic stuff later. You can probably also re-use work by others for some platforms. There is already cargo deb for making debian packages for example.

[ForLoveOfCats]

With the renewed interest in this with the looking ahead to cargo-druid I'd like to give some input on Raph's thoughts on Linux distribution. Flatpak and Snap are still very young and definitely not the primary ways to package and install programs on Linux for the near future. Perhaps eventually, I don't have a crystal ball. Currently distribution packages, portable executables w/assets all wrapped up in a tar.gz (or other compressed folder formats), and recently AppImages are the mainstays for most programs.

[luleyleo]

I use Flatpak pretty much for every app on my laptop and think it offers many advantages, so I would like to put it on the list of package formats to support.

[xStrom]

Flatpak and Snap are still very young and definitely not the primary ways to package and install programs on Linux for the near future.

I know next to nothing about these, but being young and not widely used aren't super convincing arguments in this context. They certainly have some weight, but the whole existing dependency tree of druid has those same two properties.

Given that we're designing druid for the future, we shouldn't be too afraid to use fresh solutions if they make sense.

Are they too buggy? Do they lack critical features? Do they restrict application behavior or performance too much? Are the developers egomaniacs? - These are the kinds of questions I would ask.

---

As I noted in #1047 I'm also a bit confused about the exact scope here. I think this issue here is only for collecting all the files needed for the app and not about package management, given that is a subset of #1047.

[ForLoveOfCats]

I should clarify that I'm not arguing for not having Flatpak and Snaps as officially supported packaging solutions for druid applications. Instead I'm arguing against focusing our efforts on them early on. They are not some dependency, they are software distribution solutions so how widely used they are is important for how early we push to have solid support for them. I was responding to Raph's question But possibly it's better to target flatpak and/or snap. I'd certainly want input from people in the Linux community on this. with the very anecdotal evidence that I and most Linux users I know rarely use Flatpak and Snaps and I rarely if ever see applications point to either of them as their preferred method of installation. In short my point is simply that we should focus our early efforts on what will cover the most usage cases, mainly playing nicely with distro package managers (which should be easy to expand to Flatpak and Snap in the future afaik) and a plain executable with assets/files in a subdir next to it which can easily be tar-ed up and distributed.

Flatpak and Snap do currently have some issues associated with them (such as integration issues, update/package sizes, Snap's forced auto-updates, ect) but that is out of scope for druid's eventual support of them.

I agree that this probably is better to be discussed under https://github.com/xi-editor/druid/issues/1047. However I wanted to give some late feedback to Raph's question in this thread.

[luleyleo]

I'm arguing against focusing our efforts on them early on.

I think we should keep them at least in mind, because Flatpak and Snap with their sandboxing, permission system and strict guidelines are a bit of a different beast than traditional packages.

I rarely if ever see applications point to either of them as their preferred method of installation.

Some examples that would come to my mind: Most (new) Gnome apps advertise their Flatpak, Gnome is building an entire distribution around Flatpak and elementaryOS is moving their store to it.

Snap is mostly being pushed down Ubuntu users throats by only making snaps easy to install (no terminal required). That aside Snap is more designed for Servers and IOT than desktop applications while Flatpak is explicitly designed for the desktop and would thus be of greater interest for druid.

Flatpak and Snap do currently have some issues associated with them

Best example that comes to my mind is that you can't open a directory through a dialog in a sandboxed Flatpak app.

---

So regarding Raph's question: I agree with @ForLoveOfCats that we should have good support for disto package formats such as deb or rpm, but I think it is also likely that druid users would want to publish as Flatpak because those can be easily distributed without having to convince every distro you care for to package your app, or distribute them through some side channel such as AUR or LaunchPad.

Also note that Debian for example only packages an older version of Rust which we don't even support, not sure if they would even publish an app in their stable release that would have to statically link its own version of Rust, but I might be wrong about this as I'm not particularly up-to-date about Debian.

[cmyr]

Resource Bundles

Resource bundles are a mechanism for accessing various resources (such as images, sounds, localization files, or fonts) that are used by your druid application.

Importantly, this is not about creating packaged application bundles for distribution; that is a separate (though related) topic. This means that things like the app icon, manifest, or other platform-specific assets are out of scope.

Goals

The first main goal is to have a platform-agnostic API, in druid-shell, for retreiving certain types of resources at runtime. In particular, this should abstract away the actual file system; the user should not deal with directories or files directly.

Additionally, in druid, we should offer certain static guarantees about the existence of assets, their type, and their validity. There are a number of possible approaches here, but the one that is most appealing to me is based on code generation; basically for your druid crate myapp, we generate at compile time a crate myapp-resource-bundle, that exposes in some way the actual assets that you have included. This will be tightly integrated with the cargo-druid tool (#1047); there will be a defined directory structure for storing assets, and we will scan those directories, (also taking special instructions from a manifest) and package assets up into a bundle.

The basic idea is that we can know exactly what images or icons are included with your app, and at compile time we can ensure that those assets exist and can be read, and then at runtime we can provide an interface that lets you retrieve those assets in an ergonomic, type-safe way.

druid-shell

druid-shell will provide a simple API for retrieving 'files' by name. This should be as simple as possible; dealing with things like caching or locale handling will be the responsibility of the druid-shell consumer, e.g. druid itself.

As currently imagined, this API should be a thin abstraction over the file-system. At the most simple, it might look something like,

trait Bundle {
    fn resource_for_key(&self, key: &str) ->  Result<Vec<u8>, BundleError>;
}

Where key is essentially a filename. The expectation is that in general assets will be stored in a single base location on disk, although the intention is also that they could be stored in anything with a key/value interface.

This is a dumb API. All it does is allow the user to retreive files from some location on each platform. It doesn't put the files there, it doesn't validate them, it doesn't really do anything but take some key and look for it in some location.

nesting and keypaths

The simplest possible version of this would involve saving everything in a single namespace/folder, but it's possible we will want to support nested collections/folders. In this world, key might become a 'keypath' (and maybe just actually a Path). This is mostly a detail, and I think it will shake out in the design.

first step / resources during development

As a first step (before we do too much work on creating app packages) it may make sense to have an initial implementation of this bundling idea that is just backed by a folder in my_project/target/druid-resources or something. We will need to populate this folder with assets, which feels like the job of cargo-druid, so a minimal version of bundling and a minimal version of cargo-druid will likely need to be developed together.

druid (and druid-bundler)?

All the fun stuff happens in druid.

The main idea is to have some abstraction that lets the user retrieve assets in a type-safe, failure proof way.

validating and setting up the bundle

When you build your druid app (with cargo-druid), we will load and validate all of your resources. This will include doing things like ensuring that each locale has the same set of strings, and that other files can be loaded and parsed into the expected formats.

In addition, we will have some mapping from the files in your druid project folder to the keys in your resource bundle. This is a detail to be determined, but it might involve things like appending a locale where appropriate.

codegen

Overview

The most interesting part of all of this, design-wise, is using code generation to provide a nice typed API at runtime.

At a high level, we will read your resources and write out a number of Rust source files that describe them. This will all be validated when the bundle is created, and so there will be a strong guarantee that the named resources exist, are of the correct type, and can be loaded.

The exact structure of the generated code is tbd, but my current inclination is that we will generate a crate, and then expose various modules on that crate. This will let all the generated code live in one place, and be managed entirely by the bundling system; the alternative would be to generate certain specific files inside the user's crate, which poses challenges.

A major motivation for the final design will be coming up with something that works with tools like rust-analyzer; we want the user to be able to reference their resources easily during development.

In terms of what code exactly we generate, I currently imagine two separate things; generation of the definitions of LocalizedStrings, and then generation of keys (like Env keys) for resources of other types.

Codegen for localized strings

We will generate a strings module, which will contain LocalizedStrings corresponding to the strings in your app's Fluent ftl files. These will be given names based on the name used in the fluent file, so that (say) a fluent key main-menu-title would become const MAIN_MENU_TITLE: LocalizedString = ....

different ftl files could be put in different submodules, with matching names.

Codegen for other resources

For things like images, sounds, and fonts, we will use a mechanism similar to how druid's [Env] works. For each 'asset group' (see below) we will create a typed key that can be used to retrieve the asset at runtime.

As an example: if you have an image "signup-button.png", we will create a key like,

const SIGNUP_BUTTON: Key<Image> = Key::new("signup-button.png");

and then at runtime you can retrieve the asset in question via the Env, and it will be loaded from disk, parsed, cached, and returned.

In the example above, ideally the thing returned is an actual piet::Image, ready to be drawn.

This mechanism will support common asset types, and we will write the loading code for these. If the user wishes to store data of an unknown type, this will be possible as well, and they will either have to deal with the bytes themselves, or they will be able to provide a deserialization function. This might involve using a trait and having keys of different types; the exact details will be figured out in the work.

A Sketch

the root directory of a druid project, my-app:

├── Cargo.toml
├── Druid.toml
├── src
│   ├── main.rs
# this is all generated
├── my-app-bundle
│   ├── Cargo.toml
│   ├── build.rs
│   ├── src
│   │   ├── lib.rs
│   │   ├── strings.rs

The Cargo.toml dependencies you start out with:

[dependencies]
druid = "0.6.0"
resources = { path = "my-app-bundle", package = "my-app-bundle" }

And fetching some image asset in a paint method:

fn paint(&mut self, ctx: &mut PaintCtx, data: &T, env: &Env) {
    let image = env.get_resource(&resources::NEW_USER_ICON);
    ctx.draw_image(&image, resources.NEW_USER_ICON.size, InterpolationMode::Bilinear);
    // or something like that
}

Caching

Loaded assets will be cached; I haven't thought about this a lot except to say that I think we will have some tunable cache size (in bytes) and we will eject the last used items as we bump up against this cache.

localized and DPI-aware assets ('asset group's)

In some instances, you may have multiple different versions of a single asset; for instance you might include different versions of an image tailored to different DPI, or you might have different versions for different locales (for instance if you include an icon with a currency symbol, you would want to use a different symbol in japan than in australia).

In this case, all of these different versions of an asset would be referred to be a common key, and druid would be smart about returning the correct resource. This requires accounting for things like per-monitor DPI, as well as invalidating on locale change.

Relationship with Env

Env will have a shared handle to the resource bundle, or more likely the druid type that sits on top of the resource bundle and implements a nicer API. This will be passed down unchanged to all widgets. When a resource is requested, the env will take the current locale and DPI (and potentially other things) into account when resolving that request into a concrete asset.

Open questions

directory structures

This has so far been pretty hand-wavy about specific implementation details, because it's hard to speculate before writing code; this is intended as a high-level sketch, that will almost certainly change as work starts. In particular, we will need to determine the expected structure of the resources as they exist during development, as well as the expected structure and naming conventions for assets at runtime.

relationship with cargo-druid

It also isn't entirely clear how best to approach this alongside cargo-druid, and the creation of app packages more generally. There are a number of moving pieces, and while they're related, it should be possible to work on them separately; this is especially true if we begin with a 'debug' implementation that is based on just having a folder in the target directory that we load things from.

We may end up wanting to have a druid-bundler crate that is a stand-alone tool that does the codegen, and is a dependency of the crate that we generate; there are various possibilities

a druid helper app for managing resources

Another possible project down the road would be a utility that lets you add resources to your druid project; this would itself be built with druid of course, and could make it easier to visualize asset groups.

Is generating a crate the best choice

It may be that generating a crate isn't feasible; in particular crates that are published on crates.io cannot depend on crates that are unpublished, which might be annoying. Another reasonable alternative would be to generate a module, in its own folder in the src directory.

[xStrom]

I think we should try to avoid an extra crate, exactly for the reason you pointed out - it makes life on crates.io needlessly hard. It's also unclear to me what the benefit of a crate over a module is.

[konkers]

I like the design and I see where you are going with the compile time assertion of asset existence. However, it precludes being able to dynamically load asses like I do in my app. I've had to do a lot of copying of core widgets and modifying them to be more dynamic (like Image). This approach seems like it would push druid further down the path of needing to create separate widgets and systems for more dynamic content.

I, however, don't have a great suggestion on how to reap the benefits of the compile time assertions and still allow dynamism without making every widget handle two cases.

[zbraniecki]

Hi all, we recently finished implementing quite robust localization resource manager for Fluent - https://github.com/mozilla/l10nregistry-rs

It is now used in Firefox https://bugzilla.mozilla.org/show_bug.cgi?id=1660392

It does more than most managers will need, but it ties nicely to fluent-fallback which is a higher level long-lived class that can be used to localize UIs - https://bugzilla.mozilla.org/show_bug.cgi?id=1613705

We use the combination of l10nregistry-rs and fluent-fallback to power Localization class and then extend it to DOMLocalization for DOM Fragments and from there to DocumentL10n for per-UI-document localization contexts.

This is likely a good model to consider and you may implement a simpler manager (for example in case you don't want to support live hotplugging of language packs), but shaped after it.