python 3.8 build requirements

[Phillip-May]

The build scripts for python 2.7.10 and 2.7.18 worked without a problem with Emscripten version 1.39.18 and 2.0.2, however neither worked on the python 3.8 demo. Are there additional build requirements for the wasm build of python 3.8? I tried using gcc and clang 10.0.0.4 as the host compilers but I always get an error during the Emscripten step of python.sh.

[Beuc]

Hi. What error do you get precisely?

[Phillip-May]

Starting from a clean ubuntu 20.04 LTS install, I installed all the python3 build dependencies I installed quilt I installed and activated the latest emcc/em++ version (2.0.2 (ec6d3d463ad5b709d5645466fa2ae0f62e1d468f)) I pulled this repository I then moved into the 3.8 repository and ran python.sh

During the emscripten function of python.sh (after python is done building:

running install_scripts copying build/scripts-3.8/2to3-3.8 -> /home/user/python/3.8/destdir/bin copying build/scripts-3.8/idle3.8 -> /home/user/python/3.8/destdir/bin copying build/scripts-3.8/pydoc3.8 -> /home/user/python/3.8/destdir/bin changing mode of /home/user/python/3.8/destdir/bin/2to3-3.8 to 777 changing mode of /home/user/python/3.8/destdir/bin/idle3.8 to 777 changing mode of /home/user/python/3.8/destdir/bin/pydoc3.8 to 777 rm /home/user/python/3.8/destdir/lib/python3.8/lib-dynload/_sysconfigdata_emscripten.py rm -r /home/user/python/3.8/destdir/lib/python3.8/lib-dynload/pycache /usr/bin/install -c -m 644 ../Misc/python.man \ /home/user/python/3.8/destdir/share/man/man1/python3.8.1 if test ! -d /home/user/python/3.8/destdir/lib/pkgconfig; then \ echo "Creating directory /lib/pkgconfig"; \ /usr/bin/install -c -d -m 755 /home/user/python/3.8/destdir/lib/pkgconfig; \ fi if test -f /home/user/python/3.8/destdir/bin/python3 -o -h /home/user/python/3.8/destdir/bin/python3; \ then rm -f /home/user/python/3.8/destdir/bin/python3; \ else true; \ fi (cd /home/user/python/3.8/destdir/bin; ln -s python3.8 python3) if test "3.8" != "3.8"; then \ rm -f /home/user/python/3.8/destdir/bin/python3.8-config; \ (cd /home/user/python/3.8/destdir/bin; ln -s python3.8-config python3.8-config); \ rm -f /home/user/python/3.8/destdir/lib/pkgconfig/python-3.8.pc; \ (cd /home/user/python/3.8/destdir/lib/pkgconfig; ln -s python-3.8.pc python-3.8.pc); \ rm -f /home/user/python/3.8/destdir/lib/pkgconfig/python-3.8-embed.pc; \ (cd /home/user/python/3.8/destdir/lib/pkgconfig; ln -s python-3.8-embed.pc python-3.8-embed.pc); \ fi rm -f /home/user/python/3.8/destdir/bin/python3-config (cd /home/user/python/3.8/destdir/bin; ln -s python3.8-config python3-config) rm -f /home/user/python/3.8/destdir/lib/pkgconfig/python3.pc (cd /home/user/python/3.8/destdir/lib/pkgconfig; ln -s python-3.8.pc python3.pc) rm -f /home/user/python/3.8/destdir/lib/pkgconfig/python3-embed.pc (cd /home/user/python/3.8/destdir/lib/pkgconfig; ln -s python-3.8-embed.pc python3-embed.pc) rm -f /home/user/python/3.8/destdir/bin/idle3 (cd /home/user/python/3.8/destdir/bin; ln -s idle3.8 idle3) rm -f /home/user/python/3.8/destdir/bin/pydoc3 (cd /home/user/python/3.8/destdir/bin; ln -s pydoc3.8 pydoc3) rm -f /home/user/python/3.8/destdir/bin/2to3 (cd /home/user/python/3.8/destdir/bin; ln -s 2to3-3.8 2to3) if test "x" != "x" ; then \ rm -f /home/user/python/3.8/destdir/bin/python3-32; \ (cd /home/user/python/3.8/destdir/bin; ln -s python3.8-32 python3-32) \ fi rm -f /home/user/python/3.8/destdir/share/man/man1/python3.1 (cd /home/user/python/3.8/destdir/share/man/man1; ln -s python3.8.1 python3.1) if test "xupgrade" != "xno" ; then \ case upgrade in \ upgrade) ensurepip="--upgrade" ;; \ install|) ensurepip="" ;; \ esac; \ _PYTHON_PROJECT_BASE=/home/user/python/3.8/build/Python-3.8.3/emscripten _PYTHON_HOST_PLATFORM=emscripten PYTHONPATH=/home/user/python/3.8/build/Python-3.8.3/emscripten/build/lib.emscripten-3.8:../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_emscripten python3.8 -m ensurepip \ $ensurepip --root=/home/user/python/3.8/destdir/ ; \ fi Traceback (most recent call last): File "/home/user/python/3.8/build/Python-3.8.3/Lib/decimal.py", line 3, in from _decimal import ImportError: /home/user/python/3.8/build/Python-3.8.3/emscripten/build/lib.emscripten-3.8/_decimal.so: invalid ELF header

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/user/python/3.8/build/Python-3.8.3/Lib/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/user/python/3.8/build/Python-3.8.3/Lib/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/user/python/3.8/build/Python-3.8.3/Lib/ensurepip/main.py", line 5, in sys.exit(ensurepip._main()) File "/home/user/python/3.8/build/Python-3.8.3/Lib/ensurepip/init.py", line 210, in _main return _bootstrap( File "/home/user/python/3.8/build/Python-3.8.3/Lib/ensurepip/init.py", line 129, in _bootstrap return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths) File "/home/user/python/3.8/build/Python-3.8.3/Lib/ensurepip/init.py", line 32, in _run_pip runpy.run_module("pip", run_name="main", alter_sys=True) File "/home/user/python/3.8/build/Python-3.8.3/Lib/runpy.py", line 207, in run_module return _run_module_code(code, init_globals, run_name, mod_spec) File "/home/user/python/3.8/build/Python-3.8.3/Lib/runpy.py", line 97, in _run_module_code _run_code(code, mod_globals, init_globals, File "/home/user/python/3.8/build/Python-3.8.3/Lib/runpy.py", line 87, in _run_code exec(code, run_globals) File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/main.py", line 16, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/init.py", line 40, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/cli/autocompletion.py", line 8, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/cli/main_parser.py", line 11, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/commands/init.py", line 6, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/commands/completion.py", line 6, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/cli/base_command.py", line 21, in File "", line 259, in load_module File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_internal/download.py", line 25, in File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_vendor/six.py", line 92, in get File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_vendor/six.py", line 115, in _resolve File "/tmp/tmp94tg4fi9/pip-19.2.3-py2.py3-none-any.whl/pip/_vendor/six.py", line 82, in _import_module File "/home/user/python/3.8/build/Python-3.8.3/Lib/xmlrpc/client.py", line 135, in from decimal import Decimal File "/home/user/python/3.8/build/Python-3.8.3/Lib/decimal.py", line 8, in from _pydecimal import * File "/home/user/python/3.8/build/Python-3.8.3/Lib/_pydecimal.py", line 440, in import contextvars File "/home/user/python/3.8/build/Python-3.8.3/Lib/contextvars.py", line 1, in from _contextvars import Context, ContextVar, Token, copy_context ImportError: /home/user/python/3.8/build/Python-3.8.3/emscripten/build/lib.emscripten-3.8/_contextvars.so: invalid ELF header

[Beuc]

I'll have a look at it with the newer Emscripten. This makes me realize that the tested Emscripten version was not documented, I improved the README.md. Emscripten evolves regularly with (minor) breaking changes which can lead to compilation errors. I built last month with Emscripten 1.39.18, everything should work with that version.

[Phillip-May]

Thank you for your time and the effort in making these examples public, I'm now able to build your examples and experiment with python running on the web.

[Beuc]

For the record, I just build 3.8 with Emscripten 2.0.2 and I didn't get the "invalid ELF header" error, so I'm not sure what happened.

(though I had to replace 'python' with 'python3' in the package-pythonhome.sh, to build the webprompt demo, but that's a later, unrelated error)

Let me know if you get that error again.