Frames should be events in the model rather than just timestamps

[zeptonaut]

Right now, vsyncs are stored as timestamps on Device. If, instead, we made frames actual first-class events in the model that hang off of Device, we could add things like frameTime (currently what we call the vsync time), deadline (IIUC, the time at which the data for the frame needs to be "ready to go"), and interval (IIUC the amount of time between one frameTime and the next). All of this information already exists within Scheduler::BeginFrame which we use as one source for the vsync timestamp right now.

It's probably worth noting that we're not exactly sure how we would display the deadline or interval yet, or if they would be useful.

cc: @natduca

[natduca]

Ocne they're events, the first thing we can do is hang off everythign we know about them on the args field, make the vsyncs selectable somehow, and then when they're clicked you can see the data. That might get us 85% of the value we need for folks wanting quick debugging, but with 10% of the work of doing something really fancy like visualizing deadlines

[zeptonaut]

SGTM. Android does this by drawing a bunch of big "F" circles in their own track at the top of each process that has frames associated with it:

I'm honestly not super crazy about this UI, if only because circles make it pretty hard to tell exactly where one frame ends and the next begins.

[dj2]

@petrcermak

This was originally done to make the M's for memory, the dots for layer trees and pictures. Petr might have some ideas on how to make it better.

[natduca]

There's a pragmatism applied here at two levels.

First, @ChrisCraik tried a few other options and settled on this over time. It actually works better than the other stuff that was tried... the key issue is that rendering frames overlap: a RenderFrame has start and endtimes on an array of threads, as its starting point... but they're not discrete. E.g. you can have:

  [   frame 1: thread 1   ] [frame 2 : thread ]
      [ frame 1 : thread 2             ]            [frame 2 : thread 2]

Second, as you've seen, we have lots to do. So we do a lot of UI not in the ideal way but in a way that gets us the greatest !/$.

But, correction: vsyncs are different than frames. Can we update this bug to be about vsync events? RenderFrames are the concept of frame generation plus their associated events, whereas vsyncs are an instantaneous concept that says "we could/should begin rendering here in order to align with monitor refresh, and oh btw here are some params on that vsync in case you want it."

[zeptonaut]

I guess my thought was that vsync time was an attribute of a frame - is that wrong? I know that vsyncs happen regardless of whether the frame is ready or not. Are you saying that, because of this, vsyncs should exist as an independent concept in the model?

[ChrisCraik]

Happy to hear proposals for alternative UIs, but as Nat says, we played around a lot with it.

Vsync time isn't an attribute of a frame, because frames can map in arbitrary ways to vsyncs. (Talking entirely in Android parlance here:) Frames are kicked off by a vsync, and they have a deadline of a vsync, but many times they're intentionally long running, such as inflating a new Activity (UI screen). A particularly long running frame may have e.g. 5 vsyncs that it crosses, since it takes 300ms to create that new screen and draw it from scratch.

The opposite is also true - if the app isn't rendering anything since it's showing static content, vsyncs still exist and still pulse, but there won't be any frames. In the model today, the frame doesn't stay around as long as it's presented - they show up only when a frame is being created, since that's the latency we care about optimizing.

I like the idea of vsyncs either hanging off of a device or screen concept.

[petrcermak]

Some UI ideas from me and @skyostil to make it clear where a frame begins:

[ChrisCraik]

Nice ideas - much clearer that it's tied to the tracks below it!