erkexzcx / flightlesssomething

flightlessmango.com clone, written in Go.
https://flightlesssomething.ambrosia.one/
4 stars 0 forks source link

FPS Stability Questions #12

Open thesacredsin opened 3 months ago

thesacredsin commented 3 months ago

Hi, thanks for the great tool. Just a question can you please explain what the FPS stabiility graphs represent, and is possible to maybe add a % more/less stable metric, in other words Lets say A's FPS is 5% more stable than B. At the moment, to me atleast the metric does not say much.

erkexzcx commented 3 months ago

Hi. Thank you!

can you please explain what the FPS stabiility graphs

Specifically there are 2 sections - one for FPS (which is a "final product") and another one for Frametime (which is "root cause" for FPS values). For example, look at this FPS chart of yours uploaded benchmark:

image

You can clearly see that FPS was way more stable on green line, while blue line was always spiking up and down. Mathematically, there is a way to calculate a "stability score" which is basically a "Standard Deviation": https://en.wikipedia.org/wiki/Standard_deviation

Here is the relevant part from wiki:

image

So in short - click on these buttons to disable/enable certain metric:

image

And then you have these values.


Let me try to explain again, but differently. Look at this chart:

image

Now, you have a feeling that CachyOS has much better stability, less FPS stutters and so on. Your friend, however, says the opposite, that Nobara actually gives better stability. Now, this argument might lead to ER (emergency room) visit or permanent damage to your brain tissue if your friend is strong enough. However, a civilized approach to this argument is to use mathematical approach to calculate the actual FPS stability "score":

image

According to this data, CachyOS gives better FPS stability (less spiking in terms of FPS) in comparison to NobaraOS. Congratulations, without a physical fight you proved your theory to your friend that CachyOS has greater FPS stability in comparison to Nobara OS. :sweat_smile:


Disclaimer - in order to truly compare FPS stability, you should collect each frame instead of frame at certain intervals (e.g. 100ms). In MangoHud, interval should be set to 0, so it captures each frame. I am not sure if the same is possible in Afterburner (never tried).

This is because when using e.g. interval of 500ms, MangoHud might skip stutters from time to time, so final recorded benchmark data would not represent a reality.

These FPS stability charts were added by me when I was participating in sched-ext discussions and to prove my point that bpfland gives superior FPS stability on Steam Deck playing Cyberpunk 2077 where all other schedulers gave me stutters all over the place and made the game unplayable. On the other hand, bpfland game me "dynamic FPS lock" where there are no stutters, but game fails to reach 60 fps and constantly changing.

erkexzcx commented 3 months ago

Regarding % - good idea, but I need to re-consider the design. Maybe switch to some sort of HTML table that would in % show both FPS comparison and FPS stability comparison. Good idea actually. :)

thesacredsin commented 3 months ago

Thanks for the really great explanation. To me a % less/more stable metric would make sense to understand. Also on the summary page if the Average FPS is more than 100, it does not show correctly like here https://flightlesssomething.ambrosia.one/benchmark/332

Thanks for all your hard work on this tool!

erkexzcx commented 2 months ago

Also on the summary page if the Average FPS is more than 100, it does not show correctly like here https://flightlesssomething.ambrosia.one/benchmark/332

Fixed it, rolled v0.0.20 with this fix.

Other fixes/docs are still pending. Quite busy at the moment.

thesacredsin commented 2 months ago

Thanks! Looking good