Can't capture a frame from three.js example

[greggman]

When I inspect this example

https://threejs.org/examples/webgpu_compute_geometry.html

And pick capture. It only gets the first submit whereas I'm pretty sure there are 2 calls to submit per frame.

Maybe options to capture N rAFs or N submits or N frames?

[brendan-duncan]

Thanks for the pointer to the failed capture, I keep an eye out for ones that don't work. Frame captures are definitely problematic when there is no standard definition of what a frame is. I'll see if I can come up with something, one of your suggestions or otherwise.

I'm also reworking the recorder tool so instead of having to capture all frames starting from load, you'll be able to record a single frame at an arbitrary time. That doesn't help with defining what a frame is, just thought you might be interested.

[brendan-duncan]

I used the recorder tool to record 50 frames, and that was able to record both the compute pass and the render passes, which both happen within the same rAF, so it looks like a bug in capture where it's not capturing or displaying commands after the first submit. I'll get it fixed up.

[brendan-duncan]

Ok, this one will require some thinking.

The inspector capture requestAnimationFrame handler checks to see if the original frame function is async or not. It'll wait for the async promise to resolve before considering the frame finished for the async case.

The main Three.js rAF function, specified in Animation.js _init, is not async. It calls the apps frame function, but doesn't await the app frame function.

webgpu_compute_geometry.html has an async function animate() frame function, in which it awaits computeAsync then calls renderer.render.

So the capture doesn't know to wait for the promise to resolve, it thinks after the frame function has been called the frame is finished.

Really, the three.js update function should be async, looking at the return of the app frame function and await it if the frame function is async.

But since I can't control apps doing the right thing, how do I know there were hidden promises to wait for within the frame, if the frame function given to requestAnimationFrame isn't itself async?

[brendan-duncan]

The specific reason this demo jumps out of the rAF callback before the render happens, is because three.js computeAsync is called from the rAF animate callback, with an await, and within computeAsync it calls await this.backend.resovleTimestampAsync, which is what jumps out of the rAF before it gets to the render part of the code. But since the main rAF callback in three.js (in Animation.js _init methid, doesn't return the promise returned by the animationLoop callback, intercepting the rAF callback doesn't get that promise. It would probably be an easy fix for three.js to be a better async citizen by changing if (this.animationLoop !== null) this.animationLoop(time, frame); to if (this.animationLoop !== null) return this.animationLoop(time, frame);

I still need to figure out if I can deal with this even when programs don't properly return their rAF promises...

[greggman]

that's why I suggest letting you pick N rAFs etc...

Another idea is start at rAF and end on whatever event calls getCurrentTexture. But, N rAFs seems more robust

[brendan-duncan]

getCurrentTexture wouldn't work, there are too many scenarios where that would fail. I'll add the capture n rafs thing for now since I can't think of an actual good solution. I wanted to add multiframe captures, and at some point save/load/diff captures but I haven't figured out how to create more time yet.

[brendan-duncan]

I pushed a first attempt at multi-frame capture. The render passes get caught in the second frame capture due to the unhandled promise. It feels like it could be made better, but the kids are getting restless.

[brendan-duncan]

Another thought is it can continue capturing commands after the raf callback, and if there are commands that happen before the next raf, then those belong to the previous frame. This should be able to capture async commands without the promise, because the next raf shouldn't happen until the previous frame has finished doing its async stuff.

[greggman]

That assumes they're rendering every rAF. Some sites render every other rAF ...

[brendan-duncan]

There are also people not using rAF, but there's only so much I can do, people will always find a way to break every assumption. Maybe in that case "capture everything for N milliseconds" or something. In any case, the "capture N frames" that's in there now will at least help.

[brendan-duncan]

And by "people will break all assumptions", Unity is no different. If you set a target frame rate to something not divisible from 60, it will use a timer interval instead of rAF to provide that timing. Or if it's divisible from 60, like 30, then it will use rAF but skip frames.

That would break all my assumptions for the capture and recorder tools. I just never bother to try and debug that. The only way I can think to capture the timer interval case, is to do something like capture everything for a period of time. I'm not sure what other tools, like spector, do to define what a frame is.

[greggman]

that's little strange to me given there are monitors with refresh rates of 72, 75, 144, etc all not divisiable by 60 or 30

[brendan-duncan]

It's not just strange to you, it's a long standing Unity bug. It gets brought up a couple times a year, we just never get to it.

[brendan-duncan]

I did bring it up with the web platform team again just a couple days ago. Jukka will likely tackle it when he gets back from summer break.