OpenGL shader bug?

[ghost]

I'm seeing very weird behavior with RPI apparently alpha-blending the EGL framebuffer with the dispmanx layers behind it (on Raspbian that's the Linux console with the shell).

Working shader, which renders a solid, opaque deep shade of green:

#version 100
precision mediump float;
precision mediump sampler2D;
#define texture texture2D
varying vec2 texcoord0;
varying vec2 texcoord1;
varying vec2 texcoord2;
varying vec2 texcoord3;
varying vec2 texcoord4;
varying vec2 texcoord5;
uniform mat3 colormatrix;
uniform vec3 colormatrix_c;
uniform sampler2D texture0;
uniform vec2 texture_size0;
uniform mat2 texture_rot0;
uniform vec2 pixel_size0;
uniform sampler2D texture1;
uniform vec2 texture_size1;
uniform mat2 texture_rot1;
uniform vec2 pixel_size1;
uniform sampler2D texture2;
uniform vec2 texture_size2;
uniform mat2 texture_rot2;
uniform vec2 pixel_size2;
#define LUT_POS(x, lut_size) mix(0.5 / (lut_size), 1.0 - 0.5 / (lut_size), (x))
void main() {
vec4 color = vec4(0.0, 0.0, 0.0, 1.0);
color.r = 1.000000 * vec4(texture(texture0, texcoord0)).r;
color.g = 1.000000 * vec4(texture(texture1, texcoord1)).r;
color.b = 1.000000 * vec4(texture(texture2, texcoord2)).r;
// color conversion
color.rgb = mat3(colormatrix) * color.rgb + colormatrix_c;
vec4 tmp = color;
color = vec4(tmp.r * 0.0, 0.2, 0, 1.0);
// color mapping
gl_FragColor = color;
}

Not working, the console is very visible on top of the "video":

#version 100
precision mediump float;
precision mediump sampler2D;
#define texture texture2D
varying vec2 texcoord0;
varying vec2 texcoord1;
varying vec2 texcoord2;
varying vec2 texcoord3;
varying vec2 texcoord4;
varying vec2 texcoord5;
uniform mat3 colormatrix;
uniform vec3 colormatrix_c;
uniform sampler2D texture0;
uniform vec2 texture_size0;
uniform mat2 texture_rot0;
uniform vec2 pixel_size0;
uniform sampler2D texture1;
uniform vec2 texture_size1;
uniform mat2 texture_rot1;
uniform vec2 pixel_size1;
uniform sampler2D texture2;
uniform vec2 texture_size2;
uniform mat2 texture_rot2;
uniform vec2 pixel_size2;
#define LUT_POS(x, lut_size) mix(0.5 / (lut_size), 1.0 - 0.5 / (lut_size), (x))
void main() {
vec4 color = vec4(0.0, 0.0, 0.0, 1.0);
color.r = 1.000000 * vec4(texture(texture0, texcoord0)).r;
color.g = 1.000000 * vec4(texture(texture1, texcoord1)).r;
color.b = 1.000000 * vec4(texture(texture2, texcoord2)).r;
// color conversion
color.rgb = mat3(colormatrix) * color.rgb + colormatrix_c;
vec4 tmp = color;
color = vec4(tmp.r * 0.01, 0.2, 0, 1.0);
// color mapping
gl_FragColor = color;
}

Sorry for the redundant uniforms. As you can see, in both cases gl_FragColor.a will be 1.0, i.e. opaque. The EGL dispmanx layer was created with

    VC_DISPMANX_ALPHA_T alpha = {
        .flags = DISPMANX_FLAGS_ALPHA_FROM_SOURCE,
        .opacity = 0xFF,
    };

and GL_BLEND is disabled.

This issue is especially visible if the video is letterboxed - then the regions cleared by glClear() are opaque, while the regions drawn over by the video are not.

This can be seen with mpv (git master). Normally it does color.a = 1.0;. mpv has pretty complex shader generation; this patch touches the affected code: http://sprunge.us/ghUd (if you change 0.01 to 0.0 the background console will disappear).

In both shader dumps I pasted above, the normal video shader was slightly edited for test purposes. In normal operation, the problem becomes visible if the mmal video overlay is disabled and shaders are used for video - then the video appears faded and gray, with the background console being visible.

I suspect OSD rendering might have similar problems, but haven't confirmed.

Version infos:

Linux raspberrypi 4.4.22-v7+ #912 SMP Mon Sep 26 19:04:59 BST 2016 armv7l GNU/Linux

Sep 21 2016 13:13:29 
Copyright (c) 2012 Broadcom
version 2eaf52cc53435b5ce67253af1487f9a4f9f96e2d (clean) (release)

All on Raspbian... I ran rpi-update before making a final test.

[pelwell]

You'll have more chance of getting timely support if you provide a small, self-contained test application that demonstrates the problem.

[ghost]

Is there a skeleton for creating such somewhere?

[popcornmix]

https://github.com/raspberrypi/firmware/tree/master/opt/vc/src/hello_pi/hello_triangle2 perhaps?

Does:

glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
glEnable(GL_BLEND);

help?

[ghost]

Yes, that does help. Do you still need a test program?

[popcornmix]

Not if you are happy with the behaviour. I had noticed that Kodi does that (one all GLES platforms) https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGLES.cpp#L455-L456

[ghost]

The kodi code enables it only under alpha<255, where they probably actually want to blend it with the background.

In mpv, we only want to write opaque alpha to the framebuffer, nothing else. The framebuffer itself still needs dispmanx-blending enabled for use with video overlays.

Wouldn't always enabling blending be slower?

It certainly is a bug, though.

[kuon]

I got the same issue. It is fixed by using:

    VC_DISPMANX_ALPHA_T alpha = { DISPMANX_FLAGS_ALPHA_FROM_SOURCE | DISPMANX_FLAGS_ALPHA_FIXED_ALL_PIXELS, 
        255,
        0 };

and pass &alpha to vc_dispmanx_element_add

DISPMANX_FLAGS_ALPHA_FIXED_ALL_PIXELS seems to fix it.

[Ruffio]

@popcornmix @kuon should the fix be implemented?

[kuon]

@ruffio Implemented where?

[Ruffio]

@kuon if your suggestion is the right medicine, then it should be implemented in firmware and this issue closed.

[kuon]

My fix is for the user side of the problem. I don't know if there is an actual bug in the firmware or if this is a documentation problem.

[ghost]

The bug is still a bug.

[Ruffio]

@popcornmix do we have a confirmed bug, or what does it take to get confirmed?