PPB in the browser

[AstraLuma]

One of the things we've been kicking around is PPB in the browser, because you absolutely cannot beat the browser-based distribution model.

Both pyodide and brython look active, current, and promising. (Personally, brython more so.)

It would be the "simple" matter of making a new backend based on web APIs.

[pradyunsg]

In the off-chance that this was not known for some reason: pyodide has the weight of Mozilla behind it, and they've already got the scientific Python stack running on it with some success.

https://hacks.mozilla.org/2019/04/pyodide-bringing-the-scientific-python-stack-to-the-browser/

[AstraLuma]

So, there's two big use cases:

• Browser-based demos/development
• Distributing games developed locally as browser-based.

Pyodide looks trivial for the first, but I feel like we'd have to tear apart into components to do the second?

Brython would be really great for the second, but the first would require building up a Dev UI.

I'd prefer to not support both, with the different-but-similar backends.

[AstraLuma]

And lots of schools are chromebook-based. While you can get a regular Linux environment on chromebook, I'm not sure it's something I would ask of a beginner programmer.

So maybe one of the long-term strategic goals should be a cloud-based development environment? That's... ooph, there's a lot that needs to happen to enable that.

Ok, there's 2~3 services that we'd like (as wishlist items) to happen:

• A place to publish games, so students can share with others without the whole desktop distribution quagmire
• A place to develop games, for students that don't (or can't) have a local development environment
• A place to quickly try PPB

[AstraLuma]

Note that while running Linux apps on chromebooks is possible, it's not universal, the entry point is a terminal, and the user is left setting up a full development environment from apt-get. Additionally, because it saves data locally I expect it's not compatible with a lot of institutions laptop flow, and I think I've heard rumor that dropping your entire environment is fairly easy to do.

[AstraLuma]

Oh, and some of the other options:

• Transcrypt (GitHub)
• Skulpt (Only does its own turtle stuff, not generic browser-python interop)
• PyPyJS (GitHub)

Mozila made their own because C extension interop. We don't have that problem, so we're free to explore options.

[AstraLuma]

oh, another use case:

• Mini-editor+demo for exercises embedded in online materials.

[pathunstrom]

We're definitely going to be aiming for this. We don't have a good label for it, but it's getting priorities.

[glyph]

I've been thinking about games a bit recently and I just came by to file this issue myself, and I'm thrilled to see it already exists :).

Brython already has an (apparently abandoned) proof of concept port of PyGame: https://github.com/brython-dev/brython-pygame

Just some notes from my own unfortunately abortive forays into browser-python:

• it seems to me that pyodide is too focused on the scientific stack and CPython compatibility to ever be an environment that will efficiently run an actual Python application; they'll tolerate truly unbearable load times and complex development dependencies (make, docker, gfortran, and openssl to name a few). It's still not clear to me how to separate the core "run a python program" from hundreds of megabytes of numerical fortran nonsense that is deeply uninteresting for simple games. of all the other options here it's probably the closest to being able to move into this space, but I don't think the maintainers have the interest.
• transcrypt isn't python. it's a python-like skin for javascript that can adopt various levels of verisimilitude depending on configuration options, but it doesn't seem to have an interest in passing the Python test suite.
• skulpt makes browser-python interop way too hard, and thus pitches itself as a teaching tool rather than a general purpose python-in-browser thing. however, I believe it's the technology https://anvil.works uses for its frontend stuff, so it might be possible to warp it to this end
• pypyjs is an abandoned proof of concept (python 2 only)
• pyjs is similarly abandoned

brython has its flaws, but fundamentally is trying to integrate with the PyPI ecosystem, pass the python test suite, and provide a pythonic development experience including the ability to use libraries and stuff. so it would make the most sense to me to target brython right now.

[AstraLuma]

Ok, wow, I did not notice the initialization time of pyodide. If you go to the demo notebook and hit ⏩, it can spend a full minute in just "initializing" before (presumably) and python code runs.

Thank you for the insight, @glyph! The brython console seems to load quickly and is not unreasonably slow.

[pmp-p]

I did not notice the initialization time of pyodide.

True but should not be a problem when they switch to CPython 3.8.2 + wasm, as demonstrated here https://pmp-p.github.io/panda3d-next/py3/ only first time load is heavy after that the CPython vm is in wasm compiled cache and you'll have fast startup.

[AstraLuma]

Hm, second load is better, but still not super great? I'll keep that in mind when I start doing spikes.

[pmp-p]

@astronouth7303 i could be wrong but i don't think PPB will take as long as whole Panda3D sdk to load but it's bundled in the sample. if you are interested i can do a split

Without the game engine it could be as fast as https://pmp-p.github.io/micropython-next/dom-rw.html ( only a python vm there )

brython is great, but it does not support refcounting https://github.com/brython-dev/brython/issues/275 : it can be a problem when trying to interface with compiled modules , and faster startup is only apparent because of lazy loading of modules.

[gideongrinberg]

Just to give my two cents, Skulpt, Transcrypt, and PyPyJs are either limited or slow. The same goes for Brython. What we might want to go for is WebAssembly. It supports SDL, and RenPy did it. Pyiodie is absolutely huge. I think your use cases are definitely good ideas. I might take a go at this myself.

[glyph]

What we might want to go for is WebAssembly.

WebAssembly is not a python interpreter. Are you talking about a build pipeline that builds CPython for a WebAssembly target? What did RenPy do?

It looks like RenPy did go for an Emscripten/CPython target, which is probably even slower than a pure-JS option. See for example the list of caveats on https://github.com/renpy/renpyweb#performances — Python in the browser is going to be very slow no matter what right now, and WebAssembly is not magically going to make it go fast

[gideongrinberg]

Yes, I believe that RenPy went for a CPython or Cython target. It actually has good performance, though it is 2D only. WebAssembly is actually pretty fast. I think RenPy is powered by PyGame, and @beuc ported that to SDL. Emscripten supports optimized SDL for the web. It is probably worth looking into.

[Beuc]

Hi, I'd say check for yourself how fast it is: https://renpy.beuc.net/ https://renpy.beuc.net/play/?game=the_question.zip and the Python port :) https://github.com/python-emscripten/python https://www.beuc.net/python-emscripten/demo/ RenPyWeb uses Ren'Py's very own pygame_sdl2, but I have experiments with pygame.

[gideongrinberg]

Thanks for commenting Beuc! I didn't mean to ping you, sorry. Does your system use Cython or CPython? Also, did you basically just compile Python to the web? Another quick question, do you think it would be feasible/more performant to write a Python-to-LLVM IR compiler that then works with Emscripten? I saw something like that, and I would be curious to know your thoughts.

[glyph]

@Beuc thanks, wow, that is a very cool demo! And at least on my very fast desktop, performance is quite good, nothing like the skips & jumps described in the README :). It would be very cool if you could separate these tools from Ren'Py to allow for general distribution of Python projects as web-accessible artifacts; even if the UIs were restricted to PyGame at first, this would allow for MUCH easier distribution of Python GUIs!

[Beuc]

@Gideon357 This can use CPython and (static) Cython modules. There are various Python approaches to Emscripten but usually only CPython works correctly with C and Cython modules. @glyph I'll update the README because perfs have been improving. I've been meaning to make things generic (hence the separate Python repo, even if it annoys Ren'Py's author somewhat ;)), but this requires time, so let me point https://www.patreon.com/Beuc :) Are there other game authors interested?

[pathunstrom]

@Beuc other game projects that might be interested: https://github.com/pythonarcade/arcade Arcade is pyglet based, so probably furthest from where you are now. And Pygame proper might be interested in a solution as well.

There's also https://github.com/pokepetter/ursina but they're panda3d based.

(And I'll make sure to get on your patreon when I have a few minutes.)

[gideongrinberg]

@Beuc I did a project similar to yours at some point, and your repo being generic would have been great! I think you actually have a generic toolchain for Python 2.7 but it doesn't compile on Mac. I'll get on your Patreon as well. Please let me know if you need any help with making your repo generic.

[gideongrinberg]

I do believe that RenPy has been optimized though @glyph. It includes a custom library and things like that. Once I finish my current issue, I might want to take this on.

[pmp-p]

There's also https://github.com/pokepetter/ursina but they're panda3d based.

Panda3D runs on the web for 5 years already and can use c-Python(R) 3.8.5 wasm too, so they could check https://opencollective.com/panda3d too if they want special support.

Sadly I've heard pyodide(fastcomp) developpement is going through a slowdown, but anyway i still think c-Python(R) is big and that using asyncify for games is not the good way ( 20x slowdown vs native instead of 2x wasm-jit normal cost ), but I also think rewriting everything with asyncio can be boring.

It may need to get technical to find equilibrium beetween size - speed - ease of use, so I'm inviting anyone interested in Python(any) on the browser(emsdk-wasm) or WASI(clang-10+) to freely discuss available options here https://gitter.im/Wasm-Python/community including @brettcannon because neither RenPyWeb or Panda3D web extensions were mentionned in the worth reading https://snarky.ca/what-is-the-core-of-the-python-programming-language article.

[gideongrinberg]

@Beuc isn't your repo in the @python-emscripten org generic Python compiled to WebAssembly? I just took a quick peek, but it seems to work after modifying your use of dirname

[brettcannon]

@pmp-p unless I'm missing something it seems RenPyWeb and Panda3D are CPython compiled using enscripten and not specifically a different approach to trying to solve the Python-in-the-browser problem. I think it would be great if the various projects that are compiling CPython for WebAssembly got together to have a single solution.

For me personally, though, I am digging a bit deeper to see what the right balance would be for download and performance should be while tackling even deeper issues in Python itself (e.g. lack of compliance test suite for the language). But that's a multi-year project, so getting something going with enscripten now is still a worthwhile effort IMO.

[pmp-p]

@brettcannon from what i know renpyweb/panda3d/pyodide are all using cpython official sources. But in different ways so it would be nice to settle and compare ( or even mix ! like my wapy experiment which has two vm running sync/async interleaved ).

From my diggings I got that there are four different approaches to run Python in the browser with the same codebase ( ie always using c-Python as a parser/compiler for bytecode correctness ):

• runing cpython vm in a pre-emptible wasm vm ( emscripten asyncify - renpyweb ? ) .
• running cpython with asyncio loop stepping via requestAnimationFrame ( good for accurate rendering / perf / responsiveness, Panda3D)
• running cpython vm from cpython wasm but async ( with e.g. a variation of nedbat's byterun => good for pyodide but tad slow )
• running cpython bytecode or its loop in some complex javascript engine ( asm.js-pyodide / batavia / eventually brython+byterun ).

Or one can also wait very patiently for BigInt and threading in the browser and just recompile pyodide for wasm but yeah it could be "multi-year" in that case, let alone the fact that it's hard to find a wasi browser polyfill for current spec outside some profit company.

edit/ references: byterun cpython vm : https://github.com/nedbat/byterun/ batavia vm (archived) https://github.com/beeware/batavia brython vm: https://brython.info/gallery/gallery_en.html an outdated Panda3D demo but py3 runs the same : http://rdb.name/panda3d-webgl/editor.html

( outside of scope wapy that is neither cpython or micropython https://pmp-p.github.io/wapy.es6/ see brython clock sample )

[gideongrinberg]

From my diggings I got that there are four different approaches to run Python in the browser with the same codebase ( ie always using c-Python as a parser/compiler for bytecode correctness ):

• runing cpython vm in a pre-emptible wasm vm ( emscripten asyncify - renpyweb ? ) .
• running cpython with asyncio loop stepping via requestAnimationFrame ( good for accurate rendering / perf / responsiveness, Panda3D)
• running cpython vm from cpython wasm but async ( with e.g. a variation of nedbat's byterun => good for pyodide but tad slow )
• running cpython bytecode or its loop in some complex javascript engine ( asm.js-pyodide / batavia / eventually brython+byterun ).

Another thing we might want to look at is non-CPython implementations. The obvious thing would be Cython, but it has been previously mentioned that that is bad UX. We could implement some kind of automatic compilation to Cython, then WASM (see #178) as WASM's C support is good, and Cython is well featured. I also saw a Python-to-LLVM IR compiler, which would work well with Emscripten. Finally, I've been looking at @rustpython/rustpython, which is still in its infancy but could be a good all-around packaging solution at some point.

[AstraLuma]

We are 0% married to CPython. What we do want is the capability to run Python source in the browser.

The reason for this is that as a non-profit unlikely to be making much money, having the user run their own code locally would be a big operational help.

[glyph]

Another thing we might want to look at is non-CPython implementations. The obvious thing would be Cython, but it has been previously mentioned that that is bad UX.

I'm a little confused about the repeated mentions of Cython as an alternative to CPython here. CPython is a python interpreter; Cython is a Python-like language for writing your own C extensions. It depends on the CPython runtime and, at least practically speaking, can't be used on its own without not just the CPython C runtime but a heaping helping of actual Python code as well. It's a way to optimize hotspots, not an alternative runtime.

Alternatives to Cython would be, for example, mypy-c or cffi, not CPython. Alternatives to CPython would be PyPy or Brython.

[gideongrinberg]

Another thing we might want to look at is non-CPython implementations. The obvious thing would be Cython, but it has been previously mentioned that that is bad UX.

I'm a little confused about the repeated mentions of Cython as an alternative to CPython here. CPython is a python interpreter; Cython is a Python-like language for writing your own C extensions. It depends on the CPython runtime and, at least practically speaking, can't be used on its own without not just the CPython C runtime but a heaping helping of actual Python code as well. It's a way to optimize hotspots, not an alternative runtime.

Alternatives to Cython would be, for example, mypy-c or cffi, not CPython. Alternatives to CPython would be PyPy or Brython.

My apologies. However, using Cython to compile Python to C would allow for easier emscripting, would it not? Cython can also be a Python intepreter if you use it that way. You are able to exclude the Pyrex syntax to which you are refering and produce C.

[AstraLuma]

The use of Cython sounds like a useful performance optimization to keep in mind, but not necessary to get a Python game running in the browser.

[glyph]

My apologies. However, using Cython to compile Python to C would allow for easier emscripting, would it not?

No, not really. It adds steps to the build process, edge cases that users have to be aware of, and naive usage of Cython (i.e.: if you don't add lots of type annotations and write code in a way that is fast on Cython, avoiding complex method dispatch and other Python idioms) it runs about the same speed as interpreted Python (since it has to do all the same work Python bytecode needs to do anyway due to the possibility of runtime changes) but also has the downside of increasing your executable size.

It's a great tool when you need what it does, but nothing about what it does makes this especially easier.

[pmp-p]

@Gideon357 | WASM's C support is good

True, but sadly that does not apply to everyday C code that runs on full blown OS+threads, and even less to C that cython could help produce.

wasm vm in the browser ( where it is in fact a coroutine) does not allow long running loop or blocking I/O. That's a another trap to add to those glyph just well uncovered about cython.

I'm also quite sure that trap is present in any python->C transpiler already around as it is for python code itself when it is not using a custom vm.

[AstraLuma]

Tossing this on the pile: https://github.com/RustPython/RustPython/ (WASM as a primary target)

[AstraLuma]

https://github.com/ethanhs/python-wasm