Browserify artifact deprecation

richvdh commented 1 year ago

We expose a "browser version" of the js-sdk, which is a browserified version of the entire js-sdk. It's not very well documented, but it is mentioned at https://github.com/matrix-org/matrix-js-sdk#in-a-browser.

I think we should reconsider whether we want to continue to support this artifact, for several reasons:

Most urgently: without further action, the browserify artifact is going to lose support for E2E encryption once the transition to Element R (https://github.com/vector-im/element-web/issues/21972) is done and we drop support for the legacy libolm based encryption.
It's barely tested, so is easy to break without our noticing and hard to fix when it does. (In particular, it's not what we use in-house: instead we use webpack to package element-web via typescript modules, which isn't a documented usage at all.)
Wasm dependencies for the js-sdk - such as the rust e2e library - have to be base64-encoded into javascript (increasing their size) and loaded with custom loaders: browserify doesn't have good support for Wasm, whereas webpack, node (with --experimental-wasm-modules), and jest (from v29.3) all support it natively. (There is wasmify, but I failed to get it to work and it gave a strong impression of being essentially unused.)
Frankly, browserify is outdated. Its last release was 2.5 years ago and nobody really uses it any more. More modern tools like webpack and rollup which consume libraries like the JS-SDK as ESM modules are much more flexible and powerful.

Some background on the current E2EE support: the browserify bundle doesn't include libolm; instead, if you want to use E2EE, you need a separate <script> tag to load the libolm library (from the artifacts available in gitlab; libolm exposes both a two-file version using wasm and a single-file asm.js version). That means that the matrix-js-sdk doesn't have a hard dependency on libolm, with the consequences:

Every application that wants to use E2EE - including those using webpack - ends up with custom code to load libolm (see element-web for example)
Every application that wants to use E2EE - including those using webpack - has a whole load of (barely tested) logic to handle the case that E2EE is not available, and fall back gracefully if not.

I am keen to avoid these scenarios in the transition to Element R.

On the other hand, including all the E2EE code in the browserify bundle would bloat it significantly (see https://github.com/matrix-org/matrix-js-sdk/issues/3182).

For now, I have preserved the status quo by stubbing out the rust code altogether (https://github.com/matrix-org/matrix-js-sdk/pull/3187), but the upshot of that is - as I said earlier - if we do nothing, the browserify artifact is going to lose support for E2E encryption once we drop libolm.

One potential solution is to expose two browserify bundles - one with E2EE enabled, and one without. (Or, potentially, a main one and an auxilliary one). However, that would require a bit of work to create, document, and test, and it still leaves us with artifacts that are hard to test and - as above - I don't really think that browserify is the future.

richvdh commented 1 year ago

(previously on this subject: https://github.com/vector-im/element-web/issues/11876)

Johennes commented 1 year ago

Here are download numbers for the browserified assets from v23.0.0:

Screenshot 2023-03-03 at 15 56 52

Since we also release on npm, this doesn't tell the full story of course but it at least hints at usage being fairly low.

If you are currently a user of the browser version of matrix-js-sdk, we'd like to hear about your use case to better inform our decision about whether or not to deprecate this release channel.

reivilibre commented 1 year ago

As someone who has used this file for a few small projects, it's quite convenient to have a JS file you can just drop in and then use the Matrix SDK without having to get deep into JavaScript packaging tools. I guess it's not really an important use case, but it's possible I'm not the only one. (Would it be replaced with another equivalent? I can only speak for myself but I'm not too fussy about 'single file' but I appreciated not having to process the files — just download them and include them.)

t3chguy commented 1 year ago

Would it be replaced with another equivalent?

Nope, this proposal is to cull the complexity of generating & maintaining such a bundle

bradjones1 commented 1 year ago

If you are currently a user of the browser version of matrix-js-sdk, we'd like to hear about your use case to better inform our decision about whether or not to deprecate this release channel.

I am porting my React Native application which uses the SDK, to web. (Using React Native Web.) I came across this issue in the course of trying to pare down my bundled JS size... which I think is very much related to the original intent of providing this packaging.

Screenshot from 2023-03-09 16-35-39

This is a screenshot of my bundle analysis... Matrix SDK is by far the largest dependency. I am not using E2EE and so I think I can exclude that directory from packaging in a rather blunt way... but it is still "large." (Gzipped it's 153k.)

I am rapidly coming up to speed on my JS/TS in the course of this project... however webpack and other bundling tooling is a really dark art. I'm curious how I might be able to reduce the initial-load overhead of pulling in the SDK.

tomo564 commented 1 year ago

We work on Angular webapp projects with js-sdk. Just importing the SDK and calling createClient() doesn't seem to work because of runtime errors (it looks like another browserify issue.) Sourcing the "browser" js solves the issue immediately. We'd move to the "node version" if the problem is fixed or having a reasonable workaround. Partially related, building an app in the way described below makes it a bit too large. We'd want to keep using the browser version if it continues to be much smaller.

% ng new mx-test
% cd mx-test
% npm i matrix-js-sdk
% npm i @matrix-org/olm

import { createClient } from 'matrix-js-sdk';
...

// add it to /src/app/app.compoent.ts
client = createClient({
    baseUrl: 'http://localhost:8008',
  });

% ng serve

richvdh commented 1 year ago

@tomo564 your approach looks plausible. Please could you open a new issue with your problem, describing how you build your application, and ideally with a minimal reproducible example.

tomo564 commented 1 year ago

Done. Let me know if you need more details. #3216

Johennes commented 1 year ago

Given feedback here and after re-discussing internally we've decided to go ahead with the deprecation. An official note will be included in the release notes of the cycle starting on July 11th and we'll aim to stop releasing the artifact in October 2023.

Summarizing the issues with the browserify artifact that have led to this decsion:

Integrating the Rust Crypto SDK with the browserified artifact in a maintainable way is a non-trivial amount of work. Browserify has poor support for Wasm and dependencies need to be base64-encoded (bloating their size) before being pulled in via custom loaders.
We don't use the browserified artifact in-house and so it's barely tested and easy to break.
Despite not being used in-house, maintaining the browserified artifact complicates our webpack setup because of the custom code to load libolm and the need to cover the technically possible case that libolm is not available.
Browserify itself feels like a dated technology (last upstream release was more than two years ago) and has largely been superseded by tools such as webpack or rollup.js.

t3chguy commented 1 year ago

Despite not being used in-house, maintaining the browserified artifact complicates our webpack setup because of the custom code to load libolm and the need to cover the technically possible case that libolm is not available.

The browserify build doesn't cause this, the js-sdk supporting operation without Olm does

t3chguy commented 1 year ago

The browserify artifact has been deprecated this release cycle, scheduled for full removal in the October 10th release

richvdh commented 1 year ago

The browserify artifact has been deprecated this release cycle

Shouldn't https://github.com/matrix-org/matrix-js-sdk#in-a-browser be updated?

richvdh commented 1 year ago

(and the first line of the README which says "This SDK can be run in a browser ..." ?)

t3chguy commented 1 year ago

Yup

t3chguy commented 1 year ago

This SDK can be run in a browser ...

I mean it can... Element Web does it. We just don't ship a pre-built web bundle for you. It doesn't require any node-specific libs.

richvdh commented 1 year ago

True enough I suppose. The problem is really that the "In Node.js" section conflates "node.js" and "using the SDK in anything with JS modules"

Purgenta commented 7 months ago

@t3chguy Any guidance on how someone might actually prebuild the web bundle themselves?

t3chguy commented 7 months ago

@Purgenta using a tool like Vite, Webpack, Rollup, or Browserify, look up Vite's library mode.

matrix-org / matrix-js-sdk

Browserify artifact deprecation #3189