Open colinbdclark opened 5 years ago
Just looking through the source code quickly, it looks like a aconite.compositor
only distributes the necessary aconite.readinessNotifier
grade to its direct children. Either we should modify this in Bubbles to distribute the grade to all aconite.drawable
components globally, or rearrange the component relationships so that the compositor has a reference to all drawables in the application. The former seems preferable.
And a quick sketch of Bubbles' current component tree structure:
After looking more closely into Aconite's implementation, its current readiness implementation is relatively effective if it's used properly, and if the shader only attempts to read from samplers that have texture data associated with them.
Currently, the numLayers
model value is relayed directly to the shader from the Layer Stack component. This only tells us when a new layer has been added to the composition, not when it is ready to be composited. Instead, we should create a new piece of model state for each layer that tracks whether its video is ready (wired in from aconite.video.events.afterVideoLoaded
) and then lenses this to a count of "ready layers" that is provided to the shader. This will ensure that we never read from a sampler that doesn't yet have a working video attached to it.
To track this correctly, I'll need to listen for changes to video.model url
; any change to this implies that the video is not yet ready. We know definitively that it's ready when its onVideoLoaded
event fires. It might be best if this latter event is modelized in Aconite.
In Bubbles, there is only one video per layer and the layers are tracked in a dynamic component model on the layer stack, which is relayed to the video's model, this should be fairly straightforward. When a video's isReadyToPlay
(or whatever) model changes, the compositor's numReadyLayers
should be appropriately incremented or decremented.
So, it looks like there are two issues at play here:
bubbles.compositor
until at least one layer is fully ready.layerSamplers
uniform array, even though unready samplers will never be read from. For example, if there are two layers that have successfully loaded, the first two spots in layerSamplers
should provide valid texture numbers (e.g [0, 1 ... ]) , whereas all the subsequent ones should point to a texture that has already successfully been loaded (e.g. [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]). This appears to be required even though the shader will never read from a sampler for an unready layer.Alternatively, we could immediately load an all-black image or short video loop into each texture so that there is never a situation where a texture is unbound.
I'm going to defer addressing this issue further for now in favour of implementing some new features. It will be slightly easier to address after Aconite's uniform-handling code is improved to remove the unnecessary extra array-wrapping and incomplete modelization.
Currently, the Bubbles compositor is configured to play automatically when it is created. This, however, is too soon, as no videos will have been loaded yet, and thus we get WebGL texture errors as a result. Instead, we should ensure that appropriate readiness events are fired by the
aconite.video
component(s) when they are assigned a proper URL and are playable. Aconite should handle this for us, but I think it's a question of either ensuring that the readinessNotifier is working correctly, or by binding up appropriate listeners.