Partial rendering occurs when scrolling content with texture collection

[jzolezzi]

When scrolling hubs vertically within our application, partial rendering can occur. On chrome this is a white screen. Our hubs are a vertical list of horizontal items. The user can scroll through the hubs up and down and through items within the hub right and left. We put the memory pressure value to 12 million pixels.

Texture collection occurs while scrolling up and down repeatedly through the hubs. We noticed the partial rendering intermittently. Sometimes it will say partially rendered until you hit a key to scroll or focus another item.

I wasn't able to create a quick sample but happy to share a link to our application privately.

[jzolezzi]

Not always but sometimes I see this log message when the issue reproduces.

WebGLStateManager.mjs:586 WebGL: INVALID_OPERATION: bindFramebuffer: attempt to use a deleted object
$bindFramebuffer @ WebGLStateManager.mjs:586
_bindRenderTexture @ WebGLCoreRenderExecutor.mjs:116
_execQuadOperation @ CoreRenderExecutor.mjs:52
_processQuadOperation @ CoreRenderExecutor.mjs:40
execute @ CoreRenderExecutor.mjs:28
_performRender @ CoreContext.mjs:100
_render @ CoreContext.mjs:90
render @ CoreContext.mjs:41
drawFrame @ Stage.mjs:260
lp @ WebPlatform.mjs:47
requestAnimationFrame (async)
lp @ WebPlatform.mjs:48
requestAnimationFrame (async)
lp @ WebPlatform.mjs:48 repeated 31 more times.

WebGLStateManager.mjs:636 WebGL: INVALID_OPERATION: bindTexture: attempt to use a deleted object
$bindTexture @ WebGLStateManager.mjs:636
draw @ DefaultShader.mjs:59
_renderQuadOperation @ WebGLCoreRenderExecutor.mjs:80
_execQuadOperation @ CoreRenderExecutor.mjs:64
_processQuadOperation @ CoreRenderExecutor.mjs:40
execute @ CoreRenderExecutor.mjs:28
_performRender @ CoreContext.mjs:100
_render @ CoreContext.mjs:90
render @ CoreContext.mjs:41
drawFrame @ Stage.mjs:260
lp @ WebPlatform.mjs:47
requestAnimationFrame (async)
lp @ WebPlatform.mjs:48
requestAnimationFrame (async)
lp @ WebPlatform.mjs:48 repeated 31 more times.

[jzolezzi]

I put some additional logging in WebGLCoreRenderExecutor.mjs's _bindRenderTexture. And whenever I hit this added log message the rendering issue appears.

Added it before this line: https://github.com/WebPlatformForEmbedded/Lightning/blob/10105cb33ac0b4be63c8ab36e157c318a99856b6/src/renderer/webgl/WebGLCoreRenderExecutor.mjs#L116

        if (!gl.isFramebuffer(this._renderTexture.framebuffer)) {
            console.log(`invalid frame buffer used!!!`);
        }

[g-zachar]

Hi @jzolezzi

Thanks for reporting an issue. Quick clarification: from what I understand, you're having partial rendering problem on your platform, but it also doesn't work properly on desktop (white screen)? A link would help indeed, you can e-mail me it.

Best regards

[jzolezzi]

Thanks @g-zachar, yes the issue reproduces on the platform and desktop (chrome). Email sent with the link and repro steps.

[jzolezzi]

When using renderToTexture: true and a shader the partial render issue reproduces. If I set renderToTexture: false it does not but the shader doesn't work correctly.

This issue looks similar to: https://github.com/WebPlatformForEmbedded/Lightning/issues/73

[g-zachar]

Hi @jzolezzi

While I didn't encounter issues you did while browsing your app, WebGL errors were sometimes apparent (the effect may differ depending on hardware used) and they are indeed related to resource management. I will investigate it.

Best regards

[jzolezzi]

Hi @g-zachar, I was able to create a sample based on your lightning SDK. I can see the console reporting use of deleted textures with renderToTexture is set to true.

com.plex.texture.collect.issue.zip

[g-zachar]

Hi @jzolezzi

Thanks for sharing example code. It looks like issue you're encountering is different from what I observed in our apps. I'll continue investigation and keep you updated.

Best regards

[jzolezzi]

Thanks @g-zachar Wondering if maybe we could workaround somehow? Maybe in your apps you're building up the element tree differently?

We are generally doing this in our item posters/tiles and other elements that use shaders. We have an outer element with renderToTexture: true that contains the element with the shader which in turn contains the rest of the content.

// Psuedo code
class MediaPoster extends lng.Component {
  static _template() {
    return {
      RoundedContainer: {
        w: 400,
        h: 300,
        renderToTexture: true,
        RoundedCornersGroup: {
          shader: {type: PlexRoundedCorner, radius: 6}
          RoundedCornersContent: {
            w: 400,
            h: 300,
            // Other elements here
            ...
        },
      }
    }
  }
}

[g-zachar]

Hi @jzolezzi

I found that synchronising GC with rendering frame works much better than it current solution. However this is significant change and may not go upstream without prior tests, please check it and see if it solves your problem (available here)

Best regards

[jzolezzi]

Hi @g-zachar

That branch fixes the deleted texture issue but we're running into in out of memory issue with this change pretty quickly. We narrowed it down to using render canvas2d to texture along with renderToTexture: true. lng.Tools.getShadowRect will do a render canvas2d to texture.

With your branch AND if we avoid canvas2d to texture all the rendering issues go away and we can use renderToTexture: true with shaders. So what we have done is not create a shadow rect but use shadow images instead as a workaround.

FOCUS :[R]#1:[3]Content:[1]ScreenHost:[0]Host:[0]#542:[0]HubsList:[0]#544:[12]#748:[1]HubItemsList:[0]#891:[0]#1230
2ndmind.duckdns.org/:1 [.WebGL-0xb81c9880]GL ERROR :GL_OUT_OF_MEMORY : glTexImage2D: 
2ndmind.duckdns.org/:1 [.WebGL-0xb81c9880]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete
250[.WebGL-0xb81c9880]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawElements: framebuffer incomplete
Application.mjs:125 FOCUS :[R]#1:[3]Content:[1]ScreenHost:[0]Host:[0]#542:[0]HubsList:[0]#544:[11]#739:[1]HubItemsList:[0]#890:[0]#1180
2ndmind.duckdns.org/:1 [.WebGL-0xb81c9880]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete
consoleReporter.ts:13 02/03/2020 10:06:24:488 [ruuk:b8e067f1-de81-487d-82b6-84a2d29b8a8d] INFO [Server] Testing connections to rarflix
Application.mjs:125 FOCUS :[R]#1:[3]Content:[1]ScreenHost:[0]Host:[0]#542:[0]HubsList:[0]#544:[10]#730:[1]HubItemsList:[0]#889:[0]#1130
2ndmind.duckdns.org/:1 [.WebGL-0xb81c9880]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete
Application.mjs:125 FOCUS :[R]#1:[3]Content:[1]ScreenHost:[0]Host:[0]#542:[0]HubsList:[0]#544:[9]#721:[1]HubItemsList:[0]#888:[0]#1080
2ndmind.duckdns.org/:1 [.WebGL-0xb81c9880]GL ERROR :GL_OUT_OF_MEMORY : glTexImage2D: 
2ndmind.duckdns.org/:1 [.WebGL-0xb81c9880]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete
2ndmind.duckdns.org/:1 WebGL: too many errors, no more errors will be reported to the console for this context.

Roughly how we're using create shadow rect (without the workaround).

// Psuedo code
class MediaPoster extends lng.Component {
  static _template() {
    return {
      Shadow: {
        x: 200,
        y: -20,
        mountX: 0.5,
        color: shadowColor,
        texture: lng.Tools.getShadowRect(400, 300, 6, 60, 20)
      },
      RoundedContainer: {
        w: 400,
        h: 300,
        renderToTexture: true,
        RoundedCornersGroup: {
          shader: {type: PlexRoundedCorner, radius: 6}
          RoundedCornersContent: {
            w: 400,
            h: 300,
            // Other elements here
            ...
        },
      }
    }
  }
}

[g-zachar]

Hi @jzolezzi

Can you try reducing your memoryPressure value? While older solution 'kind of' safeguarded you from exceeding certain threshold, this is not the case for this fix.

Best regards

[jzolezzi]

Thanks @g-zachar, we reduced the default memory pressure and it helps avoid the flickering. We also kept our image based shadow change in as well as we think the lng.Tools.getShadowRect makes the flickering and out of memory issue occur faster. Note this is on a specific chipset on Vizio TV. It also helps fixed the shadow getting misplaced initially as well.

[g-zachar]

Hi @jzolezzi

I'm happy that the fix improved the situation but from what you're telling there are still issues that remain unsolved, so I'm not confident enough to merge it into upstream. Let me know if you can share further details. The issue will remain open for now.

Best regards