Capability to use optimized-gemm

[jerinphilip]

Carrying forward from the previous lunch discussion, I believe there aren't any optimizations tailored to firefox enabled at the moment in this repository. This is a surprising result to me, as the extension is very usable (4 workers getting things right?). Updates are much faster than mozilla/firefox-translations variant - I can notice the updates in places, but I'm happy to see as I scroll things outside the viewport also happen to be translated.

I think this repository is currently building WASM with WORMHOLE=off:

https://github.com/jelmervdl/firefox-translations/blob/c6177dbc5516156c7a2f6471a22a0348765c0bfc/.github/workflows/publish_artifact.yml#L91

I tried grepping for wasm-gemm from browsermt/bergamot-translator@main:wasm/import-gemm-module.js but could not find any instance in this repository. What I expect this JS script to do is sense if the functions that emscripten complains to be missing show up in WebAssembly environment, if not fall back to the routines compiled by emscripten into WASM (...Fallback) This is the change I was talking about that landed in firefox-nightly recently. See Bugzilla/1722102: Import intgemm library as third_party/ sources for more details.

This repository appear to be using the simplest WASM binary (which is most likely to work on other browsers just as fine) variant (1) among those listed below.

1. Wasm Gemm : 95 wps Profiler
2. Wormhole : 390 wps (+310% to Wasm Gemm), Profiler
3. Native Firefox gemm i. SSSE3 : 490 wps (+25% to Wormhole, +415% to Wasm Gemm), Profiler ii. AVX2 : 560 wps (+43% to Wormhole, +489% to Wasm Gemm), Profiler

The above is from from Bugzilla/1746631:Add matrix multiply intrinsics for Firefox Translations#c1.

The release from here is a .xpi. We may only have intgemm ever in nightly, based on some comments spread over Bugzilla (eg: https://bugzilla.mozilla.org/show_bug.cgi?id=1722102#c16). The required changes should now be available in the current nightly. Using the attempt at import if not fallback JS script above, shall we try to use the optimized-gemm?

[jerinphilip]

This is my bad, I notice that the default is optimized gemm and which is enabled since 12 days ago and the bergamot-translator submodule has been updated to include the changes below.

https://github.com/browsermt/bergamot-translator/commit/6b2a8552345533df7e573f3ca4cc09eb3557912e

And the following patch should be getting optimized GEMM in now. https://github.com/jelmervdl/firefox-translations/blob/c6177dbc5516156c7a2f6471a22a0348765c0bfc/.github/workflows/publish_artifact.yml#L101

Is there a way of knowing the contribution of multiple workers vs this update which got in between help?

[jerinphilip]

My Windows Nightly updated today still appears to be using "fallback gemm". Have you been able to see the optimized gemm logs?

[jelmervdl]

Yep!

Is javascript.options.wasm_moz_intgemm set to true for you in about:config?

[jerinphilip]

I was in the process of filing https://github.com/browsermt/bergamot-translator/issues/360 when I saw your response. MozIntGemm appears to be set.

I can't still determine whether it's workers that are providing me with an improvement or the WebAssembly intrinsic. Given you have MozIntgemm working, may I request a quick ablation study of workers vs intrinsic if you can spare some time?

I'm also unable to see the detailed error messages in my console, do I need to do something additional to see the logs?

I assume the logs are generated from Wikipedia translating from the extension. I will try to repeat this with a linux-nightly later today.

[jerinphilip]

do I need to do something additional to see the logs? I will try to repeat this with a linux-nightly later today.

I just found the button while doing linux nightly which toggles the content messages.

Optimized GEMM appears active at both Windows and Linux for the extension (I had to update the extension, turns out I was working with an old one). Closing this issue. I'll improve the other bug reports.

Overall I'm asking a lot of stupid questions while exploring the space, I hope this is okay as I gain more understanding of the space.

Thanks for the support!