Add pyston benchmarks to speed.python.org.

[ericsnowcurrently]

This is a follow-up to #60. @pablogsal will have to do this, but I'll help, as well as shepherd this along.

[ericsnowcurrently]

This is blocked by #252 and #175.

[ericsnowcurrently]

From #60:

git clone https://github.com/pyston/python-macrobenchmarks pyston-macrobenchmarks

cat > MANIFEST.combined << EOF
[includes]
<default>
pyston-macrobenchmarks/benchmarks/MANIFEST
EOF

cat > ./cpython-perf.ini << EOF
[config]
...

[scm]
...

[compile]
...

[run_benchmark]
manifest = MANIFEST.combined
...
EOF

PYTHONPATH=./pyperformance python3 -m pyperformance compile ./cpython-perf.ini main

[pablogsal]

Ok, I will try to have this ready today or tomorrow

[pablogsal]

Why is this commented?

#git clone --branch benchmark-management https://github.com/ericsnowcurrently/python-performance pyperformance
#git clone --branch pyperformance https://github.com/ericsnowcurrently/pyston-macrobenchmarks

[ericsnowcurrently]

They were temporary commands until my PRs landed (which they have now). I copy-and-pasted too much. 🙂

[pablogsal]

I have made a first run of this and unfortunately there are some failures when building wheels (uvloop for example) due to the lack of available wheels for 3.11 and the system not having g the dependencies. For example:

2022-02-08 20:51:01,567:         CCLD     libuv.la
2022-02-08 20:51:01,567:       ar: `u' modifier ignored since `D' is the default (see `U')
2022-02-08 20:51:01,567:       building 'uvloop.loop' extension
2022-02-08 20:51:01,567:       uvloop/loop.c:212:12: fatal error: longintrepr.h: No such file or directory
2022-02-08 20:51:01,567:          #include "longintrepr.h"
2022-02-08 20:51:01,567:                   ^~~~~~~~~~~~~~~
2022-02-08 20:51:01,568:       compilation terminated.
2022-02-08 20:51:01,568:       error: command '/usr/bin/gcc' failed with exit code 1
2022-02-08 20:51:01,568:       [end of output]

or pillow:

2022-02-08 20:51:48,758:           ^^^^^^^^^^^^^^^^^^^^
2022-02-08 20:51:48,758:         File "/home/pablogsal/bench_tmpdir/venv/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
2022-02-08 20:51:48,758:           self.build_extensions()
2022-02-08 20:51:48,758:           ^^^^^^^^^^^^^^^^^^^^^^^
2022-02-08 20:51:48,758:         File "/tmp/pip-install-q16545wk/pillow_e4a1e9b1b9ac4f799bb204eeec351f46/setup.py", line 698, in build_extensions
2022-02-08 20:51:48,759:           raise RequiredDependencyException(f)
2022-02-08 20:51:48,759:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2022-02-08 20:51:48,759:       RequiredDependencyException: jpeg

or uWSGI

2022-02-08 20:53:54,226:       In file included from /home/pablogsal/bench_tmpdir/prefix/include/python3.11/Python.h:94:0,
2022-02-08 20:53:54,226:                        from plugins/python/uwsgi_python.h:2,
2022-02-08 20:53:54,226:                        from plugins/python/python_plugin.c:1:
2022-02-08 20:53:54,226:       /home/pablogsal/bench_tmpdir/prefix/include/python3.11/ceval.h:27:43: note: declared here
2022-02-08 20:53:54,226:        Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords(
2022-02-08 20:53:54,227:                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-02-08 20:53:54,227:       plugins/python/python_plugin.c: In function ‘uwsgi_python_logger’:
2022-02-08 20:53:54,227:       plugins/python/python_plugin.c:1944:17: warning: ‘PyEval_CallObjectWithKeywords’ is deprecated [-Wdeprecated-declarations]
2022-02-08 20:53:54,227:                        ul->data = (void *) PyEval_CallObject(py_getLogger, py_getLogger_args);
2022-02-08 20:53:54,227:                        ^~
2022-02-08 20:53:54,227:       In file included from /home/pablogsal/bench_tmpdir/prefix/include/python3.11/Python.h:94:0,
2022-02-08 20:53:54,227:                        from plugins/python/uwsgi_python.h:2,
2022-02-08 20:53:54,227:                        from plugins/python/python_plugin.c:1:
2022-02-08 20:53:54,227:       /home/pablogsal/bench_tmpdir/prefix/include/python3.11/ceval.h:27:43: note: declared here
2022-02-08 20:53:54,227:        Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords(

or typed ast:

2022-02-08 20:54:10,112:       ast27/Custom/typed_ast.c:81:1: warning: ‘err_input’ defined but not used [-Wunused-function]
2022-02-08 20:54:10,112:        err_input(perrdetail *err)
2022-02-08 20:54:10,112:        ^~~~~~~~~
2022-02-08 20:54:10,112:       ast27/Custom/typed_ast.c:66:12: warning: ‘PARSER_FLAGS’ defined but not used [-Wunused-function]
2022-02-08 20:54:10,112:        static int PARSER_FLAGS(PyCompilerFlags *flags)
2022-02-08 20:54:10,112:                   ^~~~~~~~~~~~
2022-02-08 20:54:10,112:       error: command '/usr/bin/gcc' failed with exit code 1
2022-02-08 20:54:10,112:       [end of output]
2022-02-08 20:54:10,112:
2022-02-08 20:54:10,113:   note: This error originates from a subprocess, and is likely not a problem with pip.
2022-02-08 20:54:10,113: error: legacy-install-failure
2022-02-08 20:54:10,113:
2022-02-08 20:54:10,113: × Encountered error while trying to install package.
2022-02-08 20:54:10,113: ╰─> typed-ast

There are many more of these failures, which is quite unfortunate because it makes the pyperformance run much slower because the attempts to compile the extensions but also fails the build.

[pablogsal]

Also, seems that there are even more tests that fail to execute even if the dependencies have been correctly installed :(

[pablogsal]

This is the final result:

10 benchmarks failed:
2022-02-08 21:26:28,085: - aiohttp
2022-02-08 21:26:28,085: - djangocms
2022-02-08 21:26:28,085: - flaskblogging
2022-02-08 21:26:28,085: - genshi
2022-02-08 21:26:28,085: - gevent_hub
2022-02-08 21:26:28,085: - gunicorn
2022-02-08 21:26:28,085: - kinto
2022-02-08 21:26:28,085: - mypy
2022-02-08 21:26:28,085: - pylint
2022-02-08 21:26:28,085: - pytorch_alexnet_inference

[ericsnowcurrently]

Yeah, see #175 (and #257). That's why I said this is blocked.

[pablogsal]

That's why I said this is blocked.

Ah, sorry about that. For some reason, I thought those were resolved 😅