Ideas to slim down the library

[cibernox]

First of all, thanks for the great library. It worked like a charm for me.

That said, i've noticed that the app is bigger than I expected it to be, around 12KB after gzip. Part of the problem is that svelte has spoiled me when it comes to the size of the bundles, if this was an angular or ember library I wouldn't bat an eye at the size.

However, it couldn't avoid notice that my 17kb gzipped app grew to 29kb app after adding svelte-i18n and ~20 translations, which is a 70% increase.

Just listing some low hanging fruit to make the library smaller, checking if you think it's worth to pursue them. I might help actually.

• Right now, the format store's value is a function that has time, date, number, capital, title, upper and lower properties attached. Because of that, rollup cannot treeshake unused features. If functions for formatting dates, times and numbers were explicit exported, rollup could just remove the unused stuff (for instance, I only format messages and times, not dates nor numbers). Arguably, capital, title, upper and lower may not even belong in this library, but I can see how they are convenient.

• Maybe getClientLocale could be something users also explicitly import if they care:

  import { getClientLocale, init } from 'svelte-i18n';
  init({
  fallbackLocale: "en",
  initialLocale: getClientLocale({ navigator: true }),
  ...
  })

• Lastly, and this is the big one really, use intl-messageformat to compile translations during the build step. That library accounts for 4/5ths of the size of svelte-i18n. If we could compile ahead of time the translations, we could not ship the library at all. We'd have to assess if the compiled JS functions, once uglified, would be bigger than the equivalent strings using the ICU message format. I reckon that they would be slightly bigger, but probably still smaller than shipping the library unless you have thousands of translations the use plural/select or interpolations. And since we already have the functions we wouldn't have to use memoize to cache the slow parsing process.

Are you interested in investigating any of the above? The last one fits very well with svelte's compiler philosophy.

[kaisermann]

Hey 👋 Thanks for your interest in making the lib better (and smaller, which is always better) 🎉 First of all, I think it's important to notice that we're possibly talking about breaking-changes here.

• Since I've first built this lib I have thought about those utility methods (title, upper, etc) and never seem to find myself using them and as you said, they're pretty easy to implement. However, I wouldn't say that removing them would reduce that much the size of the library. I'm not against removing them in the future, but would like to hear more feedback about it. It's also possible to add some kind of utility entrypoint, so devs could add them as they please.

• That's actually how it worked on v1, albeit with a little more boilerplate to set the locale. My mission withv2 was to make the API smaller and simpler. However, it's nothing that a good documentation change couldn't solve. That may actually simplify the call on server.js for sapper builds.

• Yeah, the big one, but very important indeed. Can you give an example of how you imagine that happening? Do you mean transforming the formatter calls to smaller functions which only need to receive the values to be interpolated? That would happen to all locales that you add/register at build-time? About the size issue, as you stated, I think this approach can generate a lot of weight depending on the size of the project. If and after we define what would be the result of that compilation step, we should make a benchmark to check what weight change we'd have.

Anyway, thanks again for the interest in helping the lib 😁

[cibernox]

Example of what I mean:

{
  "nearby": "Find places near your location",
  "kilometer": "{count} {count, plural, =1 {kilometer} other {kilometers}}",

could be parsed as

import { plural } from 'helpers'; 
export default {
  "nearby": "Find places near your location",
  "kilometer": count => `${count} ${plural(count, { 1: "kilometer", other: "kilometers"})}",
} 

As you can imagine, after minification that code will be

import { p } from 'helpers'; 
export default {
  "nearby": "Find places near your location",
  "kilometer": v => `${v} ${p(v,{1:"kilometer",other:"kilometers"})}"
}

As you see the overhead is typically small. You app has to have a lot of entries with special plurals and stuff for the overhead to surpass the weight of the library.

[kaisermann]

I started taking a look into linguijs which is not only much smaller (1.4kb) than intl-messageformatter, it better supports compiling the messages at build-step. I didn't went to far, since I don't have much free time right now, but will definitely continue to tackle this 😁

If we supposedly switched to lingui, for what I saw we would need/be able to do:

• ~remove deepmerge (800 bytes or so), because it only accepts shallow dictionaries. We can still support deep dictionary definitions and flat them by joining their keys. This would also remove dlv (191 bytes), the object string path parser. We could possibly do this now without being a breaking change anyway 🤔🤔🤔~ (done in v2.2.3)

• probably remove all the logic of finding the fallback locale for a certain locale: en-US to en to the defined fallback. (Can be a big breaking change if lingui behaves differently).

• lingui provides ready to use tools to ease the message compilation step. Would be nice to not have to implement them.

Or, if we want to take another route, we can use the intl-message-parser (6.5kb vs 8.6kb) instead, which is lighter and prevent some of the unused deps and code paths of intl-messageformat to end up on the lib. Or any other ICU syntax parser which is well maintained and up to date.

One of the things that is important to decide is what kind of responsibility the lib should have. Do we want to only delegate some methods and configurations to the underlying i18n lib? Or do we doing things at a lower level, i.e what I mentioned about using the parser instead of the ready-to-use formatter? I like having more control of what the lib is doing, but I'm open to feedbacks.

[cibernox]

I also discovered that what I suggested is already possible and done in react-intl: https://github.com/formatjs/react-intl/blob/master/docs/Advanced-Usage.md#pre-parsing-messages

It compiles, however, to the AST, which is a bit more verbose than the approach based in functions, and it still requires intl-messageformat (but would spare us from shipping intl-messageformat-parser).

I'm going to play a bit more with the idea of compiling translations to either ASTs or functions.

[kaisermann]

Yeah, I know! I found your issue on their repo 😆

I'll play around with the compilation step whenever possible, but for now there is #39 😁. Released as v2.2.3 🎉

Not much, but after removing dlv and deepmerge we're down from 38.7kb -> 37.1kb minified and 11.4kb -> 10.9kb.

[cibernox]

I also did a proof of concept for a babel plugin that takes the translations and converts them into functions: https://github.com/cibernox/babel-plugin-precompile-icu

Check the tests, but the tl;dr; is that it knows how to compile things like:

export default {
  nearby: "Find places near your location",
  kilometer: "{count} {count, plural, =1 {kilometer} other {kilometers}}"
};

into

import { plural } from "helpers";
export default {
  nearby: "Find places near your location",
  kilometer: count => `${count} ${plural(count, {1: "kilometer", other: "kilometers"})}`
};

And even things more complex like:

export default {
  nearby: "Find places near your location",
  kilometer: "This year { gender, select,male {he made {count, plural,=0 {no kilometres} one {one kilometre} other {{count} kilometres}}} female {she made {count, plural,=0 {no kilometres} one {one kilometre} other {{count} kilometres}}} other {they made {count, plural,=0 {no kilometres} one {one kilometre} other {{count} kilometres}}}}"
};

into

import { select, plural } from "helpers";
export default {
  nearby: "Find places near your location",
  kilometer: (count, gender) => `This year ${select(gender, {
    male: `he made ${plural(count, {
      0: "no kilometres",
      1: "one kilometre",
      other: `${count} kilometres`
    })}`,
    female: `she made ${plural(count, {
      0: "no kilometres",
      1: "one kilometre",
      other: `${count} kilometres`
    })}`,
    other: `they made ${plural(count, {
      0: "no kilometres",
      1: "one kilometre",
      other: `${count} kilometres`
    })}`
  })}`
};

As you see the overhead in size is minimal. Infact, after minification, you save space. And the more complex the message is, the more you save:

- "This year { gender, select,male {he made {count, plural,=0 {no kilometres} one {one kilometre} other {{count} kilometres}}} female {she made {count, plural,=0 {no kilometres} one {one kilometre} other {{count} kilometres}}} other {they made {count, plural,=0 {no kilometres} one {one kilometre} other {{count} kilometres}}}}"
+ (c,g)=>`This year ${d(g,{male:`he made ${p(c,{0:"no kilometres",1:"one kilometre",other:`${c} kilometres`})}`,female:`she made ${p(c,{0:"no kilometres",1:"one kilometre",other:`${c} kilometres`})}`,other:`they made ${p(c,{0:"no kilometres",1:"one kilometre",other:`${c} kilometres`})}`})}`

For now I only implemented the transformation of plural and select, but it won't be complicated to implement similar compilation for the rest of the syntax.

The next step would be to implement the helpers that I'm importing that don't yet exist, which is where I'd copy-paste from some existing library like intl-message-format, but implement the functionality as functions that rollup/webpack can tree-shake.

[cibernox]

Now I managed to write a rollup plugin that compiles the json files and imports the helpers from https://github.com/cibernox/icu-helpers

Now that the translations are functions that take arguments, the only thing that svelte-i18n would have to do, is exposes methods to register the translations and configure the preferences.

$t("my.key", { values: { count: 3 } }) should basically do

lookupLocation("my.key")({count: 3})

I think we're not too far from that.

[kaisermann]

@cibernox That's great! I'm going to check both repos soon 😉

I just created the v3 branch to start playing with some of the things mentioned here. The first things I changed were removing the casing utilities and separating the date, time and number formatters from the message one: https://github.com/kaisermann/svelte-i18n/pull/40/files#diff-31df54a9a51bd3da26df362832cc1c9fR28. Not sure if $formatDate, $formatNumber, $formatTime are too verbose... 🤔 could be $date, $number, $time, but maybe that's too generic?

Edit:

Made getClientLocale tree-shakeable too on the PR above.

[hmaesta]

When running rollup-plugin-analyzer, icu-messageformat-parser is one of the heaviest parts of my bundle.

A diet would be nice, indeed. Any update here?

[cibernox]

@hmaesta I recently released a library implementing some of this ideas: https://github.com/cibernox/svelte-intl-precompile

It has an API that I think it's almost 100% compatible with svelte-i18n. The docs say it's for SvelteKit but it will work on any rollup-based svelte app. If shouldn't occupy more than 2/3 kb after minification and compression.

I presented it on a lighting talk in svelte summit a few days ago: https://youtu.be/fnr9XWvjJHw?t=10004

[kaisermann]

Hey @hmaesta and @cibernox (loved the talk 😁!)

Unfortunately, this is not something that I can invest much time in right now. I'm open to all kinds of contribution 👀

@cibernox if you're (still) interested in "merging" the libraries, we can think about a v4 for svelte-i18n. It would be great to keep the same API and functionalities if possible though.

[hmaesta]

This page has been open for 26 days in my computer as a pinned tab, just waiting for the time when I could try @cibernox's approach. 😅

After a little struggle with Rollup, I was able to test it and save ~100 KB. 🎉

Thanks for the help. I hope it can be merged to this repository, since the name is more friendly and already have some audience.