Graph per slot plotting project PPD

[PantherX]

Concept

This idea build upon #298 where I would like to see the PPD for various projects on a particular slot presented as graph with a line for "average" PPD for the Slot and +/- 10% lines on the graph. Maybe the % can be set and you can have intervals of it to clearly see what the trend is and identify any outliers. In other words, 10 projects have PPD within +/- 5% and only 2 projects have the PPD above +/- 5% but below +/- 10%

Usage Case

It will allow Donors to see what projects are performing within normal variance and which ones aren't. This would be really helpful during Beta testing the projects to see if the variation is normal or not.

[PantherX]

I have created a mock-up in Paint 3D (really sorry for that 😢) to illustrate my concept. I hope it gets the point across:

[harlam357]

I concur that this would be a value add to the Benchmarks.

I plan for the graphing to be a bit more deterministic. Meaning, right now graphs are generated each time the user selects a different benchmark... and I think that's too aggressive (it slows things down as the user selects different benchmarks). I plan on changing the graphs to be a user selected option in benchmarks and each graph will open in its own window. How does that sound to you?

So let me make sure I understand the proposed graph. This looks like PPD over the time of the specific Project (R,C,G). The HFM benchmarks don't "think" like that... currently the benchmarks only save (and consider) the last 300 observed frames for each slot. History data doesn't "think" like this either... History considers average frame time for the entire Project (R,C,G).

What I'm thinking would be PPD per project, per slot.

Line: per Slot Y-Axis: PPD X-Axis: Project Number

This would give a clear indication of which projects perform the best or worst on a particular slot with an average line for sure... and I like the 10% and 20% thoughts but I'd first like to see some actual data before adding those.

Note that I'm talking about what HFM does today... I have plans for HFM to retain every frame time for every project for every slot. With that data available, a lot things are possible for graphing.

[PantherX]

Meaning, right now graphs are generated each time the user selects a different benchmark... and I think that's too aggressive (it slows things down as the user selects different benchmarks). I plan on changing the graphs to be a user selected option in benchmarks and each graph will open in its own window. How does that sound to you?

I haven't experienced any slow down so not 100% sure what you mean. In my workflow, on the highlighted slot, I CRTL+B to open the Benchmarks Viewer. I can then click on the tabs and it shows the graphs nicely. I have a single slot so that's by I might have instant rendering. I do like the idea of having the Graph viewer as a separate window :)

So let me make sure I understand the proposed graph. This looks like PPD over the time of the specific Project (R,C,G). The HFM benchmarks don't "think" like that... currently the benchmarks only save (and consider) the last 300 observed frames for each slot. History data doesn't "think" like this either... History considers average frame time for the entire Project (R,C,G).

Humm... that was the original idea but given what you said, would a dot work like a scatter graph so it can be visual as to how the projects compare to the average of that slot. like this:

What I'm thinking would be PPD per project, per slot.

Line: per Slot Y-Axis: PPD X-Axis: Project Number

This would give a clear indication of which projects perform the best or worst on a particular slot with an average line for sure... and I like the 10% and 20% thoughts but I'd first like to see some actual data before adding those.

That sounds cool too! I think it contains the same data as my graph above but is represented differently. If that's correct, I am fine with that too :)

Note that I'm talking about what HFM does today... I have plans for HFM to retain every frame time for every project for every slot. With that data available, a lot things are possible for graphing.

I look forward to the future :D Thanks once again for all your hard work and efforts over the years!

[harlam357]

All great ideas PantherX! Keep 'em coming! Thanks for your kind words as well!

[harlam357]

Sneak peak at how this is shaping up

[PantherX]

That looks absolutely gorgeous! So much better than my attempt using Paint/Paint 3D 😆

[harlam357]

Build 0.9.20.1256