search

faster-cpython / ideas

1.67k stars 49 forks source link

Investigate performance of the nogil branch with matching GC thresholds #593

Open mdboom opened 1 year ago

mdboom commented 1 year ago

It would be worthwhile to remove the effect of the GC threshold changes in the nogil branch by benchmarking with a version where the thresholds are the same as upstream Python, can comparing this against both upstream Python and nogil-as-it-currently stands.

mdboom commented 1 year ago

From today's meeting, it was clarified that the goal is to modify upstream to more closely match the garbage collection behavior of the nogil branch (and then benchmark and compare that to the nogil branch).

Unfortunately, it's not as simple as decreasing the number of generations and changing the thresholds. The nogil branch seems to use a dynamic threshold, so it remains to be seen how easy this will be to backport...

mdboom commented 1 year ago

I created a branch that uses a single generation with a threshold of 7000. There are far more changes to GC in nogil than that, but that's hopefully a useful approximation.

From that, you can see the effect of just increasing the garbage collection threshold vs. upstream.

The result of comparing nogil-latest against upstream with the higher gc thresholds is below. tl;dr: nogil is around 15% slower overall (mostly in single-threaded performance).

nogil-latest vs. upstream with higher gc threshold Benchmarks with tag 'apps': =========================== 2to3: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 245 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 290 ms +- 1 ms: 1.18x slower chameleon: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 6.37 ms +- 0.09 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 7.81 ms +- 0.09 ms: 1.23x slower docutils: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 2.15 sec +- 0.01 sec -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 3.15 sec +- 0.04 sec: 1.46x slower html5lib: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 57.8 ms +- 0.4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 69.2 ms +- 0.7 ms: 1.20x slower Geometric mean: 1.26x slower Benchmarks with tag 'math': =========================== float: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 60.7 ms +- 0.5 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 65.9 ms +- 0.6 ms: 1.09x slower nbody: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 92.3 ms +- 1.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 107 ms +- 2 ms: 1.16x slower pidigits: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 189 ms +- 0 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 186 ms +- 0 ms: 1.02x faster Geometric mean: 1.07x slower Benchmarks with tag 'regex': ============================ regex_compile: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 131 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 152 ms +- 0 ms: 1.16x slower regex_dna: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 210 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 205 ms +- 1 ms: 1.03x faster regex_v8: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 22.1 ms +- 0.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 21.1 ms +- 0.2 ms: 1.05x faster Benchmark hidden because not significant (1): regex_effbot Geometric mean: 1.02x slower Benchmarks with tag 'serialize': ================================ json_dumps: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 9.55 ms +- 0.28 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 10.4 ms +- 0.1 ms: 1.09x slower json_loads: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 23.7 us +- 0.3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 28.0 us +- 0.3 us: 1.18x slower pickle: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 10.2 us +- 0.1 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 10.3 us +- 0.1 us: 1.01x slower pickle_dict: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 31.2 us +- 0.1 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 30.0 us +- 0.1 us: 1.04x faster pickle_list: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 4.05 us +- 0.04 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 4.33 us +- 0.02 us: 1.07x slower pickle_pure_python: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 282 us +- 3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 321 us +- 12 us: 1.14x slower unpickle: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 13.0 us +- 0.1 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 15.5 us +- 0.2 us: 1.19x slower unpickle_list: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 4.95 us +- 0.06 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 5.20 us +- 0.03 us: 1.05x slower unpickle_pure_python: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 198 us +- 3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 229 us +- 1 us: 1.16x slower xml_etree_parse: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 122 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 137 ms +- 3 ms: 1.13x slower xml_etree_iterparse: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 80.9 ms +- 0.7 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 114 ms +- 3 ms: 1.40x slower xml_etree_generate: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 73.4 ms +- 0.9 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 83.7 ms +- 0.7 ms: 1.14x slower xml_etree_process: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 51.6 ms +- 0.2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 61.3 ms +- 0.6 ms: 1.19x slower Geometric mean: 1.13x slower Benchmarks with tag 'startup': ============================== python_startup: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 8.24 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 9.18 ms +- 0.01 ms: 1.11x slower python_startup_no_site: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 5.93 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 6.59 ms +- 0.01 ms: 1.11x slower Geometric mean: 1.11x slower Benchmarks with tag 'template': =============================== django_template: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 32.9 ms +- 0.6 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 37.4 ms +- 0.7 ms: 1.14x slower genshi_text: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 20.2 ms +- 0.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 24.4 ms +- 0.3 ms: 1.21x slower genshi_xml: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 45.9 ms +- 0.6 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 52.0 ms +- 0.4 ms: 1.13x slower mako: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 9.69 ms +- 0.08 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 13.2 ms +- 0.1 ms: 1.37x slower Geometric mean: 1.21x slower All benchmarks: =============== 2to3: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 245 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 290 ms +- 1 ms: 1.18x slower async_generators: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 338 ms +- 3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 377 ms +- 2 ms: 1.12x slower async_tree_none: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 260 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 311 ms +- 8 ms: 1.20x slower async_tree_cpu_io_mixed: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 469 ms +- 6 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 534 ms +- 9 ms: 1.14x slower async_tree_io: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 534 ms +- 4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 616 ms +- 7 ms: 1.15x slower async_tree_memoization: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 315 ms +- 3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 381 ms +- 6 ms: 1.21x slower asyncio_tcp: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 511 ms +- 3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 525 ms +- 2 ms: 1.03x slower chameleon: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 6.37 ms +- 0.09 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 7.81 ms +- 0.09 ms: 1.23x slower chaos: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 68.8 ms +- 0.9 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 74.6 ms +- 0.7 ms: 1.08x slower comprehensions: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 23.9 us +- 0.3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 26.3 us +- 0.2 us: 1.10x slower bench_mp_pool: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 24.0 ms +- 0.1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 23.8 ms +- 0.4 ms: 1.01x faster bench_thread_pool: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 792 us +- 4 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 1.64 ms +- 0.01 ms: 2.08x slower coroutines: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 26.1 ms +- 0.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 26.8 ms +- 0.1 ms: 1.03x slower coverage: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 99.2 ms +- 1.2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 109 ms +- 1 ms: 1.10x slower crypto_pyaes: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 73.7 ms +- 0.9 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 81.8 ms +- 0.5 ms: 1.11x slower deepcopy: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 332 us +- 3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 375 us +- 2 us: 1.13x slower deepcopy_reduce: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 2.97 us +- 0.04 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 3.42 us +- 0.04 us: 1.15x slower deepcopy_memo: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 33.8 us +- 0.5 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 40.1 us +- 0.2 us: 1.19x slower deltablue: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 3.11 ms +- 0.02 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 3.91 ms +- 0.05 ms: 1.26x slower django_template: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 32.9 ms +- 0.6 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 37.4 ms +- 0.7 ms: 1.14x slower djangocms: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 33.1 us +- 1.2 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 37.9 us +- 2.8 us: 1.14x slower docutils: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 2.15 sec +- 0.01 sec -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 3.15 sec +- 0.04 sec: 1.46x slower dulwich_log: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 63.5 ms +- 0.4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 68.1 ms +- 0.3 ms: 1.07x slower fannkuch: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 368 ms +- 4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 436 ms +- 2 ms: 1.18x slower float: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 60.7 ms +- 0.5 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 65.9 ms +- 0.6 ms: 1.09x slower create_gc_cycles: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 1.46 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 1.59 ms +- 0.00 ms: 1.09x slower gc_traversal: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 3.38 ms +- 0.04 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 3.67 ms +- 0.01 ms: 1.09x slower generators: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 79.1 ms +- 0.5 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 78.8 ms +- 0.2 ms: 1.00x faster genshi_text: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 20.2 ms +- 0.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 24.4 ms +- 0.3 ms: 1.21x slower genshi_xml: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 45.9 ms +- 0.6 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 52.0 ms +- 0.4 ms: 1.13x slower go: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 137 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 146 ms +- 1 ms: 1.07x slower hexiom: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 6.06 ms +- 0.05 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 6.89 ms +- 0.05 ms: 1.14x slower html5lib: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 57.8 ms +- 0.4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 69.2 ms +- 0.7 ms: 1.20x slower json: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 4.59 ms +- 0.10 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 5.06 ms +- 0.12 ms: 1.10x slower json_dumps: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 9.55 ms +- 0.28 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 10.4 ms +- 0.1 ms: 1.09x slower json_loads: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 23.7 us +- 0.3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 28.0 us +- 0.3 us: 1.18x slower logging_format: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 6.44 us +- 0.14 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 7.66 us +- 0.11 us: 1.19x slower logging_silent: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 89.6 ns +- 2.5 ns -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 104 ns +- 0 ns: 1.16x slower logging_simple: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 5.80 us +- 0.08 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 7.00 us +- 0.12 us: 1.21x slower mako: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 9.69 ms +- 0.08 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 13.2 ms +- 0.1 ms: 1.37x slower mdp: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 2.48 sec +- 0.01 sec -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 2.79 sec +- 0.02 sec: 1.12x slower meteor_contest: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 109 ms +- 4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 124 ms +- 0 ms: 1.14x slower mypy2: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 309 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 457 ms +- 9 ms: 1.48x slower nbody: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 92.3 ms +- 1.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 107 ms +- 2 ms: 1.16x slower nqueens: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 77.6 ms +- 0.8 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 85.8 ms +- 0.7 ms: 1.11x slower pickle: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 10.2 us +- 0.1 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 10.3 us +- 0.1 us: 1.01x slower pickle_dict: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 31.2 us +- 0.1 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 30.0 us +- 0.1 us: 1.04x faster pickle_list: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 4.05 us +- 0.04 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 4.33 us +- 0.02 us: 1.07x slower pickle_pure_python: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 282 us +- 3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 321 us +- 12 us: 1.14x slower pidigits: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 189 ms +- 0 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 186 ms +- 0 ms: 1.02x faster pprint_safe_repr: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 676 ms +- 4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 765 ms +- 7 ms: 1.13x slower pprint_pformat: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 1.39 sec +- 0.01 sec -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 1.58 sec +- 0.01 sec: 1.13x slower pycparser: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 1.06 sec +- 0.01 sec -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 1.15 sec +- 0.04 sec: 1.09x slower pyflate: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 403 ms +- 4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 470 ms +- 2 ms: 1.17x slower python_startup: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 8.24 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 9.18 ms +- 0.01 ms: 1.11x slower python_startup_no_site: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 5.93 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 6.59 ms +- 0.01 ms: 1.11x slower raytrace: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 286 ms +- 7 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 346 ms +- 7 ms: 1.21x slower regex_compile: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 131 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 152 ms +- 0 ms: 1.16x slower regex_dna: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 210 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 205 ms +- 1 ms: 1.03x faster regex_v8: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 22.1 ms +- 0.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 21.1 ms +- 0.2 ms: 1.05x faster richards: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 42.5 ms +- 0.8 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 48.4 ms +- 0.3 ms: 1.14x slower scimark_fft: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 309 ms +- 4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 383 ms +- 3 ms: 1.24x slower scimark_lu: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 105 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 128 ms +- 3 ms: 1.21x slower scimark_monte_carlo: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 65.2 ms +- 1.4 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 76.4 ms +- 1.0 ms: 1.17x slower scimark_sor: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 106 ms +- 3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 123 ms +- 1 ms: 1.15x slower scimark_sparse_mat_mult: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 4.12 ms +- 0.12 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 4.94 ms +- 0.01 ms: 1.20x slower spectral_norm: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 94.7 ms +- 1.0 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 112 ms +- 0 ms: 1.18x slower sqlglot_parse: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 1.38 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 1.79 ms +- 0.06 ms: 1.30x slower sqlglot_transpile: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 1.67 ms +- 0.01 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 2.10 ms +- 0.08 ms: 1.26x slower sqlglot_optimize: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 51.6 ms +- 0.3 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 57.1 ms +- 1.3 ms: 1.11x slower sqlglot_normalize: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 108 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 115 ms +- 1 ms: 1.06x slower sqlite_synth: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 2.67 us +- 0.06 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 2.62 us +- 0.03 us: 1.02x faster sympy_expand: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 455 ms +- 5 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 537 ms +- 10 ms: 1.18x slower sympy_integrate: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 20.2 ms +- 0.1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 22.8 ms +- 0.1 ms: 1.13x slower sympy_sum: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 163 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 192 ms +- 1 ms: 1.18x slower sympy_str: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 281 ms +- 2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 329 ms +- 4 ms: 1.17x slower telco: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 6.52 ms +- 0.15 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 6.92 ms +- 0.22 ms: 1.06x slower thrift: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 772 us +- 9 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 851 us +- 9 us: 1.10x slower unpack_sequence: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 43.6 ns +- 0.8 ns -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 58.1 ns +- 1.8 ns: 1.33x slower unpickle: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 13.0 us +- 0.1 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 15.5 us +- 0.2 us: 1.19x slower unpickle_list: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 4.95 us +- 0.06 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 5.20 us +- 0.03 us: 1.05x slower unpickle_pure_python: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 198 us +- 3 us -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 229 us +- 1 us: 1.16x slower xml_etree_parse: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 122 ms +- 1 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 137 ms +- 3 ms: 1.13x slower xml_etree_iterparse: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 80.9 ms +- 0.7 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 114 ms +- 3 ms: 1.40x slower xml_etree_generate: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 73.4 ms +- 0.9 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 83.7 ms +- 0.7 ms: 1.14x slower xml_etree_process: Mean +- std dev: [bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163] 51.6 ms +- 0.2 ms -> [bm-20230416-linux-x86_64-faster%2dcpython-nogil_latest-3.12.0a4-1d39009] 61.3 ms +- 0.6 ms: 1.19x slower Benchmark hidden because not significant (2): pathlib, regex_effbot Geometric mean: 1.15x slower Ignored benchmarks (5) of results/bm-20230531-3.12.0a4-0fd3163/bm-20230531-linux-x86_64-mdboom-match_nogil_gc-3.12.0a4-0fd3163.json: asyncio_tcp_ssl, dask, richards_super, tomli_loads, typing_runtime_protocols ![plot](https://github.com/faster-cpython/ideas/assets/38294/d4e81a6f-f06b-4544-85b4-6bbc0cd79b8a)

For completeness, here's the result of nogil-latest against the upstream where it branched off.