Performance wiki page?

[Starbuck5]

Description

I saw a post on reddit asking people what pygame-ce was, and the poster was skeptical of abstract claims of "more performant" because they didn't see a benchmarks page or anything to look into. I think that's totally understandable, and this is something we should have.

This page could be linked from the README as a way to back up a claim that we have performance improvements over pygame, and just to show off our performance enhancements.

My idea is a collection of small benchmark scripts run on pygame-ce vs pygame (baseline), with the machine run on and versions and dates and percent performance difference. Each highlighted area could have a short blurb explaining the "why," which is also important for architecture-specific performance improvements, like AVX2 blitters.

Performance PRs could be found by looking through the performance label. I wonder if we have any that didn't get that label?

[MyreMylar]

This is the repo CPython's 'faster-cpython' team setup to do performance benchmarking: https://github.com/faster-cpython/benchmarking-public

for some inspiration.

It might be nice to have something that does some comparison performance runs (same test for a function run on pygame-ce vs pygame) on CI - then uploads some raw data from these runs so we can turn it into nicely formatted bar chart comparisons on a web page.

As well as CE vs Pygame upstream, it would be nice to benchmark against older versions (also probably should include the SDL & Python versions used in the benchmarks).

[MyreMylar]

Was having a think about this today and ran across this Github Action, which may serve our purposes.

[Starbuck5]

I don't want this to be a live performance dashboard like pyperformance, I want it to be a manual write up of speed ups we can achieve in certain operations. Featuring code snippets and performance numbers from a certain system.

To do a live performance dashboard like that Github Action makes me think of, we need an "overall pygame benchmark," which is not something we have right now.

@oddbookworm Are you still interested in this working on this?

[Starbuck5]

@itzpr3d4t0r Would you be interested in working on this? I don't think Andrew would mind.

A lot of the optimizations were created by you, it would be nice to have a user friendly and informative page to link to when we'd like to claim "pygame-ce is faster"

Andrew started with https://github.com/pygame-community/pygame-ce/wiki/Performance-Comparisons-Against-Upstream-Pygame , which we've agreed is way too in depth. (But Andrew maintains that it is very statistically rigorous!)

[Notenlish]

Notes from conversation on discord: https://discord.com/channels/772505616680878080/772505616680878083/1280053176896323646

The performance comparison wiki page could be split into 2 categories, spesific optimizations(for users who want to learn about it in detail) and general performance speedups observed in games made for regular pygame(when its switched to pygame-ce)

It'll also be helpful to provide a guide for regular pygame users who wish to upgrade to pygame-ce in the performance comparison page(They could learn about the new additions like fblits there)

As starbuck said, it'll be better for microbenchmarks to be less in depth.