Closed alexmojaki closed 1 year ago
Here's my analysis of the difference in performance.
loadPyodide
plus the main runPython
) has gone from ~4.5s to ~2s, i.e. ~2.5s faster.loadPyodide
: 1900 msrunPython
: 2700 msloadPyodide
: 900 msrunPython
: 1200 ms/pyc/
:
loadPyodide
: 3000 msrunPython
: 1500 ms/pyc/
is critical.pyodide.asm.data
: 5.0 mbpyodide.asm.wasm
: 2.5 mbpython_stdlib.zip
: 3.6 mbpyodide.asm.wasm
: 2.9 mbloadPyodide
. The difference in file size doesn't significantly affect download time on my network.python_stdlib.zip
is significantly smaller than pyodide.asm.data
because cdn.jsdelivr.net compresses it with br
(the actual .zip appears uncompressed and is 10mb) while grist-static.com doesn't compress pyodide.asm.data
. So if we serve the files ourselves and want to optimize download speed, we need to check that they're being compressed.loadPackage
takes 700ms before and 800ms after. So the CPU time (probably to uncompress the archives) is still the main factor, and it's slightly increased by having to deal with bigger wheel files, but shouldn't matter much in the grand scheme.Timing change looks great! Thanks for the breakdown. A nice side-effect of this work may be a speed up for some situations in the desktop app also, where the default sandbox is pyodide (run from node).
Goes with https://github.com/gristlabs/grist-core/pull/603
core/sandbox/pyodide/build_packages.sh
now runspyodide py-compile
to compile.py
files inside wheels to.pyc
files which should load faster.The webworker loads pyodide itself from https://cdn.jsdelivr.net/pyodide/v0.23.4/pyc/pyodide.js, where the
/pyc/
has a similar meaning for the Python standard library. If we want to avoid using this CDN and continue serving pyodide ourselves, I don't know yet what's required to do this manually, i.e. to serve the compiled standard library.Currently, the wheel containing grist's source code is not compiled.
core/sandbox/pyodide/packages.js
now writes a file calledsandbox/pyodide/package_filenames.json
next to it. A copy of that file lives next towebworker.js
which imports it, which is slightly less weird than the previous hardcoded list. Using a symlink doesn't work (I'm not 100% sure that's not solvable though) and I haven't yet given serious thought to automating the copying.Since the compiled wheels aren't pushed to S3 yet, currently
make requirements
will download the old (now incorrect) wheels. Butbuild_packages.sh
removes the grist wheel, so you have to do a bit of an awkward dance at the moment to get everything working.My next plan is to carefully test the difference in performance this makes. Hopefully it doesn't make things worse.