Open stalgiag opened 3 years ago
I'm thinking of this in terms of physics engine update, would the expectation be that the physics stops on unfocus? Also I'm not 100% sure the animation frame pausing on unfocus behaviour is consistent enough to not cause sync issues.
From my experience (ie. may be wrong) unfocus event also fires when the window is not active but the tab is on top, in which case the animation frame do continue to execute, which again can cause sync issues. Need to test this bit to be sure if any implementation would be possible.
Should we account for this in deltaTime in some way? My instinct is no, but this isn't clearcut for me because sketches that rely on deltaTime will be very fragile to tab switching.
I agree with this. Maybe the thing to do here is add to the documentation for deltaTime to communicate this unique case?
From a quick look around the internet, it doesn't seem like there are ways to influence browser-level refresh throttling when tabs aren't foregrounded. (Unless you played audio file in the webpage, which should prevent the throttling.)
I also found that there's events tied to "Page Visibility", which could wrap in p5 so that creators could subscribe to this event if they needed to do something special to account for the expected jump in deltaTime
. Something like onWebPageHidden
onWebPageShown
? But I'm not sure how helpful that really is. What do y'all think?
I haven't come across the Page Visibility events before and a quick read through it I'm still not sure does it ties to the pausing of requestAnimationFrame
? I'm a bit more reluctant to add additional APIs like onWebPageHidden/Shown
, it would be best if this can be handled by the library itself.
Fwiw, I use three.js as well, so out of curiosity I did a test on the three.js clock which has a getDelta
method that is similar to deltaTime
in p5 and the tests I did have the same issue as p5 when pausing the animation or changing window focus, the value of getDelta()
spikes and the animation either jumps or gets stuck.
p5 has events for window blur and focus which appears to be triggered by changing the tab as well: https://github.com/processing/p5.js/blob/main/src/core/main.js#L565
That could be used to pause the draw loop or offset the value of deltaTime
. I haven't tested it much but seems to work okay. However, it's also triggered by things like opening the developer console or switching to another program window with the browser still visible.
But it seems like a pause in rendering at any point kind of makes calculating deltaTime
the way it's intended impossible? I think it might make sense in this case to set deltaTime
to 0 to prevent a big spike or a guess.
I made an attempt to fix this issue: https://github.com/owenbmcc/p5.js/commit/00d461eabd4026349056e13ba8a1f135278d7760
I don't know if this is the right approach, but it seems to fix things for the example I originally created: https://editor.p5js.org/owenroberts/sketches/LO8vidvA2
This isn't directly related to the unfocused tab issue, but the issue I first noticed with GIF animations getting delayed is also caused by using loop
and noLoop
in a sketch.
I'm not familiar with the thinking that went into the animation loop for p5, so I'm curious looking at the _draw
function in main.js, why is it that the requestAnimationFrame(this._draw)
is only called if _loop
is set to true?
https://github.com/processing/p5.js/blob/main/src/core/main.js#L409
Would it make sense to continue calling draw just to update the _frameRate
, _lastFrameTime
and deltaTime
properties and then not calling redraw()
if _loop
is false
? That would prevent the spike in deltaTime
caused by using noLoop
.
@owenbmcc I wouldn't recommend tying it to blur and focus event as you mentioned switching focus while the window is still visible will fire the blur event as well and I find that potentially unusable especially since I can't pop the developer console out and monitor behaviour there. The page visibility API as mentioned above could be a solution but without trying it I don't know what its actual behaviour is like in relation to requestAnimationFrame
The implementation of the draw loop as you can imagine goes back a long time so I can't say much about it as I don't know that much. I think the idea of making deltaTime
0 (by controlling _lastFrameTime
) is worth exploring, I'm not sure there's that much value in updating deltaTime
when noLoop()
as that will provide a non 0 value to deltaTime
when redraw()
is called which doesn't make sense to me.
@limzykenneth I agree that makes sense.
I should have explained more clearly, the solution I came up with for the unfocused tab issue wouldn't pause the sketch completely if you open the console or focus another program. My approach was to set deltaTime
to 0 only in the frame directly following a blur event, so if you switch tabs and requestAnimationFrame
is no longer running, when you return to the tab, it will have been paused, but since requestAnimationFrame
seems to continue running as long as the page is visible, it will only pause for 1 frame. I'm still not sure that it is the right approach, but just wanted to clarify.
For what it's worth, as a user I expect deltaTime
to be the actual time between frames. If something causes a big drop in framerate, I'd hope to see a big spike in deltaTime
.
Maybe the thing to do here is add to the documentation for deltaTime to communicate this unique case?
I would much prefer a warning in the documentation explaining that browsers may stop rendering animations when backgrounded, than p5 assuming that I want an artificially small/consistent deltaTime
.
@weslord I think that is a reasonable expectation as well.
I feel differently about this but not enough to advocate for a change.
In general, I do expect deltaTime
to be the actual time between frames. But I guess I don't ever expect the time between frames to be inconsistent & unpredictable. Coming from Unity, where I primarily work and the original inspiration for deltaTime
I believe, deltaTime
can be relied on to never spike in a way that breaks the code unless the hardware is incapable of running the project. If a Unity project is unfocused it is frozen (including deltaTime
). In Unity, if a frame is fired 15 milliseconds pass and then the project loses focus, deltaTime
upon regaining focus will be 15 ms. If you want it to run its update cycle when unfocused you can set the project settings to do so, but then it demands all of the necessary resources to run consistently (ie it doesn't throttle). We don't have either of these options because our hand is forced by the browser. This means that deltaTime
cannot be used in the same way that one could use it in Unity (or any game engine) because using the 'real-time' logic of deltaTime
will break even simple sketches when the browser tab is switched.
Bouncing ellipse tied to real-time
This simple sketch breaks immediately upon tab switching. This wouldn't happen if we were doing the standard x += speed
frame-by-frame because the throttling would constrain the values. But since deltaTime
isn't constrained by the throttling, the value quickly jumps the ellipse offscreen.
In my opinion, this makes the most instinctive uses of 'real time' very fragile to tab-switching. The useful connection between frame-by-frame rendering and 'real-time' is made unusable by the throttling. I am not suggesting that we do anything other than mention this in the documentation and get rid of all internal uses of deltaTime
but I do think this is a limitation that we need to work into our understanding of p5's relationship to 'real-time'.
Relates to #6785, need to test
Hi! This came up while fixing a problem with duplicate GIF timing (see discussion in #4824 ). Wanted to ask a larger question about timing and
deltaTime
here before throwing a workaround into the GIF timing.My question is: Should a coder expect there to be unpredictable spikes in
deltaTime
when coming back to a sketch after switching tabs?This sketch illustrates this. If you click into the sketch, switch tabs, and come back after a few seconds, you will probably see a number that is nowhere near 1000 next to 'deltaTime' in the console (depends on how 'busy' your CPU currently is)
I know this mostly comes down to a combination of
window.performance.now
and browser throttling behavior with unfocused tab animation callbacks.This is less of a problem with
millis()
where a coder would expect the value to correlate to the time since the sketch started. But withdeltaTime
, the animation frame throttling makes the difference in 'real-time' between frames very unexpected even if it is true to the space between frames.Should we account for this in
deltaTime
in some way? My instinct is no, but this isn't clearcut for me because sketches that rely ondeltaTime
will be very fragile to tab switching.For me, this brings up larger questions about whether we should manually throttle unfocused sketches in a controlled way by default to make behavior predictable and consistent across browsers/hardware? What do others think? @limzykenneth @montoyamoraga @outofambit @akshay-99 @owenbmcc
Most appropriate sub-area of p5.js?