pypy3 slowdown for json_dumps benchmark

[gitlab-importer]

In Heptapod by bitbucket_importer on Jan 14, 2018, 22:31

Created originally on Bitbucket by mattip

Using the cpython performance set of benchmarks, pypy3-v5.10.1 is very slow as if the JIT is off.

#setup
python -mpip install performance; python -mperformance venv create
pypy2 -mpip install performance; pypy2 -mperformance venv create
pypy3 -mpip install performance; pypy3 -mperformance venv create
# runs the bnechmark
python -mperformance run -b json_dumps --venv venv/pypy3* 
python -mperformance run -b json_dumps --venv venv/pypy2* 
python -mperformance run -b json_dumps --venv venv/cpython* 

Gives me ~5ms for pypy2, ~17 ms for cpython2, ~50 ms for pypy3

Attachments: logfile2logfile

[gitlab-importer]

In Heptapod by bitbucket_importer on Jan 14, 2018, 23:07

Created originally on Bitbucket by mattip

pypy2 logfile

[gitlab-importer]

In Heptapod by bitbucket_importer on Jan 14, 2018, 23:08

Created originally on Bitbucket by mattip

pypy3 logfile

[gitlab-importer]

In Heptapod by bitbucket_importer on Jan 14, 2018, 23:12

Created originally on Bitbucket by mattip

according to the logfile from pypy2 and the logfile3 from pypy3, the difference is in lib-python/sre_parse.py, in the Tokenizer.__next() function where we index into unicode. It gets to a frozen_structseq.__getitem__, which is not JITted.

Would the unicode-utf8 branch help here?

[gitlab-importer]

In Heptapod by @arigo on Jan 15, 2018, 09:29

Surely sre_parse is only invoked at start-up, to build the regular expressions? It shouldn't be the reason for the major slow-down.

[gitlab-importer]

In Heptapod by @arigo on Jan 15, 2018, 09:45

Also, if you got these logfiles by saying PYPYLOG=jit:logfile pypy -m performance..., then it's the logfiles of the runner process, not of the actual subprocesses that run the benchmarks...

[gitlab-importer]

In Heptapod by @arigo on Jan 15, 2018, 10:07

Running the actual subprocess with --jit off gives results that are even 5x slower, which makes it ~15x slower than CPython. I think that it's too much. Probably the problem is unrelated to the JIT, and we should be able to figure it out by inspecting a --jit off execution (gdb, randomly ctrl-c; or valgrind).

[gitlab-importer]

In Heptapod by @arigo on Jan 15, 2018, 13:11

Progress in 4e1c19053bd9 and previous checkins

[gitlab-importer]

In Heptapod by @arigo on Jan 15, 2018, 18:00

Much better. Only 20% slower than pypy2 now.