Question. Using multi pass OpenGL Shader afterRendering().

[Xuno]

We are implementing SuperXBR post process which offers reasonable quality/performance real time upscaling. The algorithm requires multi pass operations. We try to run it as a custom shader with use event afterRendering. But the question of how to use your geometry engine in custom programs/fbo/ like you use it in OpenGLVideo.cpp:

    geometry->create();
    gr.updateGeometry(geometry);

example here: https://github.com/Xuno/Xuno-QtAV/blob/testGLSuperScale/examples/player/filters/XunoGlslFilter.cpp#L275

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/40153939-question-using-multi-pass-opengl-shader-afterrendering?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github).

[wang-bin]

I'm considering to make Geometry.h and GeometryRenderer.h public, what do you think? When to support new custom vertex mappings, for example rendering to a sphere (VR), Geometry class is required.

[Xuno]

Yes, this is good news.

[Xuno]

We use a custom filter that creates its own FBO by event afterRendering(); Can you recommend how to draw back to texture from custom FBO so it displays in the video player?

Here is an example: https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/examples/player/filters/XunoGlslFilter.cpp#L41

[wang-bin]

Pass the texture rendered by superscale to renderer? You can try to add virtual GLuint frameTexture() const to GLSLFilter.h. It returns d->fbo.texture() by default. Use frameTexture() in https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/src/filter/GLSLFilter.cpp#L140 In your example, it returns your texture id (don't delete fbo everytime).

[Xuno]

Thanks for solution. We added next code you your class: https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/src/filter/GLSLFilter.cpp#L145 May be need better realize of code ? But if we try move d.fbo->texture(); to in GLuint GLSLFilter::frameTexture() const then DPTR_D(GLSLFilter); not compiled.

And added next code to our class for redefine and use virtual method: https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/examples/player/filters/XunoGlslFilter.cpp#L327 https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/examples/player/filters/XunoGlslFilter.cpp#L67 For saving texture of custom fbo during next rendering in QtAV we moved procedure clearing fbo and programs to next frame in afterRendering() event.

Questions:

1. Our custom texture of fbo has a bigger size (x2), should we decrease the size of texture to size of QtAV fbo ?
2. In the first pass we get texture from QtAV fbo, do additional passes without rotating and back to QtAV fbo again. And at the display we can see a vertically mirrored image now. May be we have different geometry rendering methods ? (https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/examples/player/filters/XunoGlslFilter.cpp#L304). For now we just rotate texture in last pass before send to fbo of QtAV.

[wang-bin]

But if we try move d.fbo->texture(); to in GLuint GLSLFilter::frameTexture() const then DPTR_D(GLSLFilter); not compiled.

Use DPTR_D(const GLSLFilter);

Our custom texture of fbo have bigger size (x2), should we decrease size of texture to size of QtAV fbo ?

FBO in GLSLFilter then should be x2 input frame size. I considered this case when writing GLSLFilter. One option is changing GLSLFilter::setOutputSize(const QSize&) to GLSLFilter::setOutputSize(qreal w, qreal h), negative w or h means scale the input width or height by factor w or h, positive value means fixed width or height, 0 means the same as input width or height.

For example, setOutputSize(-2,1080) means fbo width is x2, height is always 1080. setOutputSize(0, 0) means FBO has the same size as input frame. But negative w or h seems ugly, other solutions like setOutputSize(qreal w, qreal h, bool scaleW = false, bool scaleH = false) // w, h is positive seem ugly too, so I have not added this feature.

[wang-bin]

2.

Y is mirrored https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/src/opengl/OpenGLVideo.cpp#L115

[wang-bin]

setOutputSize(qreal w, qreal h) above is a wrong answer because it is used by GLSLFilter's fbo and fbo size is used by output frame. For your custom texture, that fbo size and output frame size are different. Maybe setOutputSize should only set output frame size at https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/src/filter/GLSLFilter.cpp#L130 https://github.com/Xuno/Xuno-QtAV/blob/GLSLFileter01/src/filter/GLSLFilter.cpp#L131 and use another function to set the fbo size(or simply always use input frame size)

[Xuno]

Use DPTR_D(const GLSLFilter);

This works.

Can you port this virtual method to your master code?

[Xuno]

setOutputSize(qreal w, qreal h)

Now we don't use any changes of outputsize by setOutputSize(), and render of QtAV correctly showing video even if texture in custom fbo has x2 dimension. But why is this ok? If videoframe is defined early by fixed video frame size from meta data:

VideoFormat fmt(VideoFormat::Format_RGB32);
VideoFrame f(d.fbo->width(), d.fbo->height(), fmt);
f.setBytesPerLine(d.fbo->width()*fmt.bytesPerPixel(), 0);

But later texture from other custom fbo is different geometry than frame size :

    GLTextureInterop *interop = new GLTextureInterop(frameTexture());
    f.setMetaData(QStringLiteral("surface_interop"), QVariant::fromValue(VideoSurfaceInteropPtr((interop))));
    *frame = f;

Is task resolved internally by OpenGL subsystem? If my custom texture is 16bits per channel (Format_RGB48) or float 32 bits (Format_RGB32f), how will texture bytes be aligned correctly in this case?

P.S. With every new video frame we have new Texture id (in my case x2 (two passes)), so its a problem with long video, oversize max value for type GLuint 2^32? It eq. /25fps/3600 = 47k/hour. 47k/2(passes) = 23k/hour. Need to use one textureid ?

[Xuno]

Qt 5.8. Windows. Should these be ignored "semantic issue"?

[Xuno]

Success, the multi pass OpenGL SuperXBR Shader is now running in version 0.2.4. Upscaling quality is significantly better. We will continue experimenting with additional post filters.

[wang-bin]

Greate. I will study your code and improve current GLSLFilter api. Current api is not flexible enough

[wang-bin]

I've made Geometry.h and GeometryRenderer.h public. Usually TexturedGeometry is enough. See OpenGLVideo. I installed your player and tested super xbr, it complains shader error.

Warning: QOpenGLShader::compile(Fragment): ERROR: 3:54: '' :   illegal recursive expansion of macro i
ERROR: 3:55: '' :   illegal recursive expansion of macro luma
ERROR: 3:55: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:60: '' :   illegal recursive expansion of macro i
ERROR: 3:60: '' :   illegal recursive expansion of macro i
ERROR: 3:60: '' :   illegal recursive expansion of macro i
ERROR: 3:60: '' :   illegal recursive expansion of macro i
ERROR: 3:61: '' :   illegal recursive expansion of macro i
ERROR: 3:61: '' :   illegal recursive expansion of macro i
ERROR: 3:61: '' :   illegal recursive expansion of macro i
ERROR: 3:61: '' :   illegal recursive expansion of macro i
ERROR: 3:62: '' :   illegal recursive expansion of macro i
ERROR: 3:62: '' :   illegal recursive expansion of macro i
ERROR: 3:62: '' :   illegal recursive expansion of macro i
ERROR: 3:62: '' :   illegal recursive expansion of macro i
ERROR: 3:63: '' :   illegal recursive expansion of macro i
ERROR: 3:63: '' :   illegal recursive expansion of macro i
ERROR: 3:63: '' :   illegal recursive expansion of macro i
ERROR: 3:63: '' :   illegal recursive expansion of macro i
ERROR: 3:64: '' :   illegal recursive expansion of macro i
ERROR: 3:64: '' :   illegal recursive expansion of macro i
ERROR: 3:64: '' :   illegal recursive expansion of macro i
ERROR: 3:64: '' :   illegal recursive expansion of macro i
ERROR: 3:74: '' :   illegal recursive expansion of macro luma
ERROR: 3:74: '' :   illegal recursive expansion of macro luma
ERROR: 3:75: '' :   illegal recursive expansion of macro luma
ERROR: 3:75: '' :   illegal recursive expansion of macro luma
ERROR: 3:76: '' :   illegal recursive expansion of macro luma
ERROR: 3:76: '' :   illegal recursive expansion of macro luma
ERROR: 3:77: '' :   illegal recursive expansion of macro luma
ERROR: 3:77: '' :   illegal recursive expansion of macro luma
ERROR: 3:78: '' :   illegal recursive expansion of macro luma
ERROR: 3:78: '' :   illegal recursive expansion of macro luma
ERROR: 3:79: '' :   illegal recursive expansion of macro luma
ERROR: 3:79: '' :   illegal recursive expansion of macro luma
ERROR: 3:80: '' :   illegal recursive expansion of macro luma
ERROR: 3:80: '' :   illegal recursive expansion of macro luma
ERROR: 3:81: '' :   illegal recursive expansion of macro luma
ERROR: 3:81: '' :   illegal recursive expansion of macro luma
ERROR: 3:82: '' :   illegal recursive expansion of macro luma
ERROR: 3:82: '' :   illegal recursive expansion of macro luma
ERROR: 3:83: '' :   illegal recursive expansion of macro luma
ERROR: 3:83: '' :   illegal recursive expansion of macro luma
ERROR: 3:84: '' :   illegal recursive expansion of macro luma
ERROR: 3:84: '' :   illegal recursive expansion of macro luma
ERROR: 3:85: '' :   illegal recursive expansion of macro luma
ERROR: 3:85: '' :   illegal recursive expansion of macro luma
ERROR: 3:86: '' :   illegal recursive expansion of macro luma
ERROR: 3:86: '' :   illegal recursive expansion of macro luma
ERROR: 3:87: '' :   illegal recursive expansion of macro luma
ERROR: 3:87: '' :   illegal recursive expansion of macro luma
ERROR: 3:89: '' :   illegal recursive expansion of macro luma
ERROR: 3:89: '' :   illegal recursive expansion of macro luma
ERROR: 3:90: '' :   illegal recursive expansion of macro luma
ERROR: 3:90: '' :   illegal recursive expansion of macro luma
ERROR: 3:91: '' :   illegal recursive expansion of macro luma
ERROR: 3:91: '' :   illegal recursive expansion of macro luma
ERROR: 3:92: '' :   illegal recursive expansion of macro luma
ERROR: 3:92: '' :   illegal recursive expansion of macro luma
ERROR: 3:93: '' :   illegal recursive expansion of macro luma
ERROR: 3:93: '' :   illegal recursive expansion of macro luma
ERROR: 3:94: '' :   illegal recursive expansion of macro luma
ERROR: 3:94: '' :   illegal recursive expansion of macro luma
ERROR: 3:95: '' :   illegal recursive expansion of macro luma
ERROR: 3:95: '' :   illegal recursive expansion of macro luma
ERROR: 3:96: '' :   illegal recursive expansion of macro luma
ERROR: 3:96: '' :   illegal recursive expansion of macro luma
ERROR: 3:97: '' :   illegal recursive expansion of macro luma
ERROR: 3:97: '' :   illegal recursive expansion of macro luma
ERROR: 3:98: '' :   illegal recursive expansion of macro luma
ERROR: 3:98: '' :   illegal recursive expansion of macro luma
ERROR: 3:99: '' :   illegal recursive expansion of macro luma
ERROR: 3:99: '' :   illegal recursive expansion of macro luma
ERROR: 3:100: '' :   illegal recursive expansion of macro luma
ERROR: 3:100: '' :   illegal recursive expansion of macro luma
ERROR: 3:104: '' :   illegal recursive expansion of macro i
ERROR: 3:104: '' :   illegal recursive expansion of macro i
ERROR: 3:104: '' :   illegal recursive expansion of macro i
ERROR: 3:104: '' :   illegal recursive expansion of macro i
ERROR: 3:105: '' :   illegal recursive expansion of macro i
ERROR: 3:105: '' :   illegal recursive expansion of macro i
ERROR: 3:105: '' :   illegal recursive expansion of macro i
ERROR: 3:105: '' :   illegal recursive expansion of macro i

Warning: *** Problematic Fragment shader source code ***
#version 130
#define lowp
#define mediump
#define highp
#line 1
//Create FBO: 3342x2086 (3342x2086)  #2
//recompiling a shader program:
//header:
#define HOOKED_raw texture0
#define HOOKED_pos texcoord0
#define HOOKED_size texture_size0
#define HOOKED_rot texture_rot0
#define HOOKED_pt pixel_size0
#define HOOKED_tex(pos) (1.000000 * vec4(texture(HOOKED_raw, pos)).rgba)
#define HOOKED_texOff(off) HOOKED_tex(HOOKED_pos + HOOKED_pt * vec2(off))
#define MAIN_raw texture0
#define MAIN_pos texcoord0
#define MAIN_size texture_size0
#define MAIN_rot texture_rot0
#define MAIN_pt pixel_size0
#define MAIN_tex(pos) (1.000000 * vec4(texture(MAIN_raw, pos)).rgba)
#define MAIN_texOff(off) MAIN_tex(MAIN_pos + MAIN_pt * vec2(off))

//texture0
uniform sampler2D MAIN_raw;

//texture pos from vertex (texcoord0)
in vec2 MAIN_pos;

//texture size (texture_size0)
uniform vec2 MAIN_size;

//texture rotation (texture_rot0)
uniform mat2 MAIN_rot;

//pixel size (pixel_size0)
uniform vec2 MAIN_pt;

vec4 superxbr() {
vec4 i[4*4];
vec4 res;
#define i(x,y) i[(x)*4+(y)]
float luma[4*4];
#define luma(x, y) luma[(x)*4+(y)]
#define GET_SAMPLE(pos) HOOKED_texOff(pos)
#define SAMPLE4_MUL(sample4, w) ((sample4)*(w))
vec2 dir = fract(HOOKED_pos * HOOKED_size) - 0.5;
dir = transpose(HOOKED_rot) * dir;
vec2 dist = HOOKED_size * min(HOOKED_pos, vec2(1.0) - HOOKED_pos);
if (dir.x * dir.y < 0.0 && dist.x > 1.0 && dist.y > 1.0)
    return vec4(0.0);
if (dir.x < 0.0 || dir.y < 0.0 || dist.x < 1.0 || dist.y < 1.0)
    return GET_SAMPLE(-dir);
#define IDX(x, y) vec2(float(x)-1.25, float(y)-1.25)

for (int x = 0; x < 4; x++)
for (int y = 0; y < 4; y++) {
i(x,y) = GET_SAMPLE(IDX(x,y));
luma(x,y) = dot(i(x,y), vec4(0.2126, 0.7152, 0.0722, 0));
}

{ // step
mat4 d1 = mat4( i(0,0), i(1,1), i(2,2), i(3,3) );
mat4 d2 = mat4( i(0,3), i(1,2), i(2,1), i(3,0) );
mat4 h1 = mat4( i(0,1), i(1,1), i(2,1), i(3,1) );
mat4 h2 = mat4( i(0,2), i(1,2), i(2,2), i(3,2) );
mat4 v1 = mat4( i(1,0), i(1,1), i(1,2), i(1,3) );
mat4 v2 = mat4( i(2,0), i(2,1), i(2,2), i(2,3) );
float dw = 0.129633;
float ow = 0.175068;
vec4 dk = vec4(-dw, dw+0.5, dw+0.5, -dw);
vec4 ok = vec4(-ow, ow+0.5, ow+0.5, -ow);
vec4 d1c = SAMPLE4_MUL(d1, dk);
vec4 d2c = SAMPLE4_MUL(d2, dk);
vec4 vc = SAMPLE4_MUL(v1+v2, ok)/2.0;
vec4 hc = SAMPLE4_MUL(h1+h2, ok)/2.0;
float d_edge = 0.0;
d_edge += -1.0 * abs(luma(2,0) - luma(0,2));
d_edge -= -1.0 * abs(luma(3,2) - luma(1,0));
d_edge += 1.0 * abs(luma(1,1) - luma(0,2));
d_edge -= 1.0 * abs(luma(2,1) - luma(1,0));
d_edge += 1.0 * abs(luma(2,0) - luma(1,1));
d_edge -= 1.0 * abs(luma(3,2) - luma(2,1));
d_edge += 2.0 * abs(luma(2,1) - luma(1,2));
d_edge -= 2.0 * abs(luma(2,2) - luma(1,1));
d_edge += -1.0 * abs(luma(3,1) - luma(1,3));
d_edge -= -1.0 * abs(luma(2,3) - luma(0,1));
d_edge += 1.0 * abs(luma(2,2) - luma(1,3));
d_edge -= 1.0 * abs(luma(1,2) - luma(0,1));
d_edge += 1.0 * abs(luma(3,1) - luma(2,2));
d_edge -= 1.0 * abs(luma(2,3) - luma(1,2));
float o_edge = 0.0;
o_edge += 1.0 * abs(luma(1,0) - luma(1,1));
o_edge -= 1.0 * abs(luma(0,1) - luma(1,1));
o_edge += 2.0 * abs(luma(1,1) - luma(1,2));
o_edge -= 2.0 * abs(luma(1,1) - luma(2,1));
o_edge += 1.0 * abs(luma(1,2) - luma(1,3));
o_edge -= 1.0 * abs(luma(2,1) - luma(3,1));
o_edge += 1.0 * abs(luma(2,0) - luma(2,1));
o_edge -= 1.0 * abs(luma(0,2) - luma(1,2));
o_edge += 2.0 * abs(luma(2,1) - luma(2,2));
o_edge -= 2.0 * abs(luma(1,2) - luma(2,2));
o_edge += 1.0 * abs(luma(2,2) - luma(2,3));
o_edge -= 1.0 * abs(luma(2,2) - luma(3,2));
float str = smoothstep(0.0, 0.600000 + 1e-6, abs(d_edge));
res = mix(mix(d2c, d1c, step(0.0, d_edge)),
    mix(hc,   vc, step(0.0, o_edge)), 1.0 - str);
vec4 lo = min(min( i(1,1), i(2,1) ), min( i(1,2), i(2,2) ));
vec4 hi = max(max( i(1,1), i(2,1) ), max( i(1,2), i(2,2) ));
res = clamp(res, lo, hi);
} // step
return res;
}  // superxbr

vec4 hook() {
    return superxbr();
}

//body:
// custom hook

void main(void)
{
gl_FragColor = hook();
//gl_FragColor = texture(texture0, texcoord0);
}

***
Debug: initShaders error (pass) 0
Warning: QOpenGLShader::compile(Fragment): ERROR: 3:50: '' :   illegal recursive expansion of macro i
ERROR: 3:51: '' :   illegal recursive expansion of macro luma
ERROR: 3:51: '' :   illegal recursive expansion of macro i
ERROR: 3:54: '' :   illegal recursive expansion of macro i
ERROR: 3:54: '' :   illegal recursive expansion of macro i
ERROR: 3:54: '' :   illegal recursive expansion of macro i
ERROR: 3:54: '' :   illegal recursive expansion of macro i
ERROR: 3:55: '' :   illegal recursive expansion of macro i
ERROR: 3:55: '' :   illegal recursive expansion of macro i
ERROR: 3:55: '' :   illegal recursive expansion of macro i
ERROR: 3:55: '' :   illegal recursive expansion of macro i
ERROR: 3:56: '' :   illegal recursive expansion of macro i
ERROR: 3:56: '' :   illegal recursive expansion of macro i
ERROR: 3:56: '' :   illegal recursive expansion of macro i
ERROR: 3:56: '' :   illegal recursive expansion of macro i
ERROR: 3:57: '' :   illegal recursive expansion of macro i
ERROR: 3:57: '' :   illegal recursive expansion of macro i
ERROR: 3:57: '' :   illegal recursive expansion of macro i
ERROR: 3:57: '' :   illegal recursive expansion of macro i
ERROR: 3:58: '' :   illegal recursive expansion of macro i
ERROR: 3:58: '' :   illegal recursive expansion of macro i
ERROR: 3:58: '' :   illegal recursive expansion of macro i
ERROR: 3:58: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:59: '' :   illegal recursive expansion of macro i
ERROR: 3:69: '' :   illegal recursive expansion of macro luma
ERROR: 3:69: '' :   illegal recursive expansion of macro luma
ERROR: 3:70: '' :   illegal recursive expansion of macro luma
ERROR: 3:70: '' :   illegal recursive expansion of macro luma
ERROR: 3:71: '' :   illegal recursive expansion of macro luma
ERROR: 3:71: '' :   illegal recursive expansion of macro luma
ERROR: 3:72: '' :   illegal recursive expansion of macro luma
ERROR: 3:72: '' :   illegal recursive expansion of macro luma
ERROR: 3:73: '' :   illegal recursive expansion of macro luma
ERROR: 3:73: '' :   illegal recursive expansion of macro luma
ERROR: 3:74: '' :   illegal recursive expansion of macro luma
ERROR: 3:74: '' :   illegal recursive expansion of macro luma
ERROR: 3:75: '' :   illegal recursive expansion of macro luma
ERROR: 3:75: '' :   illegal recursive expansion of macro luma
ERROR: 3:76: '' :   illegal recursive expansion of macro luma
ERROR: 3:76: '' :   illegal recursive expansion of macro luma
ERROR: 3:77: '' :   illegal recursive expansion of macro luma
ERROR: 3:77: '' :   illegal recursive expansion of macro luma
ERROR: 3:78: '' :   illegal recursive expansion of macro luma
ERROR: 3:78: '' :   illegal recursive expansion of macro luma
ERROR: 3:80: '' :   illegal recursive expansion of macro luma
ERROR: 3:80: '' :   illegal recursive expansion of macro luma
ERROR: 3:81: '' :   illegal recursive expansion of macro luma
ERROR: 3:81: '' :   illegal recursive expansion of macro luma
ERROR: 3:82: '' :   illegal recursive expansion of macro luma
ERROR: 3:82: '' :   illegal recursive expansion of macro luma
ERROR: 3:83: '' :   illegal recursive expansion of macro luma
ERROR: 3:83: '' :   illegal recursive expansion of macro luma
ERROR: 3:84: '' :   illegal recursive expansion of macro luma
ERROR: 3:84: '' :   illegal recursive expansion of macro luma
ERROR: 3:85: '' :   illegal recursive expansion of macro luma
ERROR: 3:85: '' :   illegal recursive expansion of macro luma
ERROR: 3:86: '' :   illegal recursive expansion of macro luma
ERROR: 3:86: '' :   illegal recursive expansion of macro luma
ERROR: 3:87: '' :   illegal recursive expansion of macro luma
ERROR: 3:87: '' :   illegal recursive expansion of macro luma
ERROR: 3:88: '' :   illegal recursive expansion of macro luma
ERROR: 3:88: '' :   illegal recursive expansion of macro luma
ERROR: 3:89: '' :   illegal recursive expansion of macro luma
ERROR: 3:89: '' :   illegal recursive expansion of macro luma
ERROR: 3:90: '' :   illegal recursive expansion of macro luma
ERROR: 3:90: '' :   illegal recursive expansion of macro luma
ERROR: 3:91: '' :   illegal recursive expansion of macro luma
ERROR: 3:91: '' :   illegal recursive expansion of macro luma
ERROR: 3:95: '' :   illegal recursive expansion of macro i
ERROR: 3:95: '' :   illegal recursive expansion of macro i
ERROR: 3:95: '' :   illegal recursive expansion of macro i
ERROR: 3:95: '' :   illegal recursive expansion of macro i
ERROR: 3:96: '' :   illegal recursive expansion of macro i
ERROR: 3:96: '' :   illegal recursive expansion of macro i
ERROR: 3:96: '' :   illegal recursive expansion of macro i
ERROR: 3:96: '' :   illegal recursive expansion of macro i

Warning: *** Problematic Fragment shader source code ***
#version 130
#define lowp
#define mediump
#define highp
#line 1

//Create FBO: 3342x2086 (3342x2086) #3
//recompiling a shader program:
//header:
#define HOOKED_raw texture0
#define HOOKED_pos texcoord0
#define HOOKED_size texture_size0
#define HOOKED_rot texture_rot0
#define HOOKED_pt pixel_size0
#define HOOKED_tex(pos) (1.000000 * vec4(texture(HOOKED_raw, pos)).rgba)
#define HOOKED_texOff(off) HOOKED_tex(HOOKED_pos + HOOKED_pt * vec2(off))
#define MAIN_raw texture0
#define MAIN_pos texcoord0
#define MAIN_size texture_size0
#define MAIN_rot texture_rot0
#define MAIN_pt pixel_size0
#define MAIN_tex(pos) (1.000000 * vec4(texture(MAIN_raw, pos)).rgba)
#define MAIN_texOff(off) MAIN_tex(MAIN_pos + MAIN_pt * vec2(off))

//texture0
uniform sampler2D MAIN_raw;

//texture pos from vertex (texcoord0)
in vec2 MAIN_pos;

//texture size (texture_size0)
uniform vec2 MAIN_size;

//texture rotation (texture_rot0)
uniform mat2 MAIN_rot;

//pixel size (pixel_size0)
uniform vec2 MAIN_pt;

vec4 superxbr() {
vec4 i[4*4];
vec4 res;
#define i(x,y) i[(x)*4+(y)]
float luma[4*4];
#define luma(x, y) luma[(x)*4+(y)]
#define GET_SAMPLE(pos) HOOKED_texOff(pos)
#define SAMPLE4_MUL(sample4, w) ((sample4)*(w))
vec2 dir = fract(HOOKED_pos * HOOKED_size / 2.0) - 0.5;
if (dir.x * dir.y > 0.0)
    return GET_SAMPLE(0);
#define IDX(x, y) vec2(x+y-3,y-x)
for (int x = 0; x < 4; x++)
for (int y = 0; y < 4; y++) {
i(x,y) = GET_SAMPLE(IDX(x,y));
luma(x,y) = dot(i(x,y), vec4(0.2126, 0.7152, 0.0722, 0));
}
{ // step
mat4 d1 = mat4( i(0,0), i(1,1), i(2,2), i(3,3) );
mat4 d2 = mat4( i(0,3), i(1,2), i(2,1), i(3,0) );
mat4 h1 = mat4( i(0,1), i(1,1), i(2,1), i(3,1) );
mat4 h2 = mat4( i(0,2), i(1,2), i(2,2), i(3,2) );
mat4 v1 = mat4( i(1,0), i(1,1), i(1,2), i(1,3) );
mat4 v2 = mat4( i(2,0), i(2,1), i(2,2), i(2,3) );
float dw = 0.175068;
float ow = 0.129633;
vec4 dk = vec4(-dw, dw+0.5, dw+0.5, -dw);
vec4 ok = vec4(-ow, ow+0.5, ow+0.5, -ow);
vec4 d1c = SAMPLE4_MUL(d1, dk);
vec4 d2c = SAMPLE4_MUL(d2, dk);
vec4 vc = SAMPLE4_MUL(v1+v2, ok)/2.0;
vec4 hc = SAMPLE4_MUL(h1+h2, ok)/2.0;
float d_edge = 0.0;
d_edge += 1.0 * abs(luma(1,1) - luma(0,2));
d_edge -= 1.0 * abs(luma(2,1) - luma(1,0));
d_edge += 1.0 * abs(luma(2,0) - luma(1,1));
d_edge -= 1.0 * abs(luma(3,2) - luma(2,1));
d_edge += 4.0 * abs(luma(2,1) - luma(1,2));
d_edge -= 4.0 * abs(luma(2,2) - luma(1,1));
d_edge += 1.0 * abs(luma(2,2) - luma(1,3));
d_edge -= 1.0 * abs(luma(1,2) - luma(0,1));
d_edge += 1.0 * abs(luma(3,1) - luma(2,2));
d_edge -= 1.0 * abs(luma(2,3) - luma(1,2));
float o_edge = 0.0;
o_edge += 1.0 * abs(luma(1,0) - luma(1,1));
o_edge -= 1.0 * abs(luma(0,1) - luma(1,1));
o_edge += 4.0 * abs(luma(1,1) - luma(1,2));
o_edge -= 4.0 * abs(luma(1,1) - luma(2,1));
o_edge += 1.0 * abs(luma(1,2) - luma(1,3));
o_edge -= 1.0 * abs(luma(2,1) - luma(3,1));
o_edge += 1.0 * abs(luma(2,0) - luma(2,1));
o_edge -= 1.0 * abs(luma(0,2) - luma(1,2));
o_edge += 4.0 * abs(luma(2,1) - luma(2,2));
o_edge -= 4.0 * abs(luma(1,2) - luma(2,2));
o_edge += 1.0 * abs(luma(2,2) - luma(2,3));
o_edge -= 1.0 * abs(luma(2,2) - luma(3,2));
float str = smoothstep(0.0, 0.600000 + 1e-6, abs(d_edge));
res = mix(mix(d2c, d1c, step(0.0, d_edge)),
    mix(hc,   vc, step(0.0, o_edge)), 1.0 - str);
vec4 lo = min(min( i(1,1), i(2,1) ), min( i(1,2), i(2,2) ));
vec4 hi = max(max( i(1,1), i(2,1) ), max( i(1,2), i(2,2) ));
res = clamp(res, lo, hi);
} // step
return res;
}  // superxbr

vec4 hook() {
    return superxbr();
}

//body:
// custom hook
void main(void)
{
gl_FragColor = hook();
//gl_FragColor = texture(texture0, texcoord0);
}

***
Debug: initShaders error (pass) 1

[wang-bin]

Debug: "GL_VERSION: 4.3.0 - Build 20.19.15.4531" Debug: "GL_VENDOR: Intel" Debug: "GL_RENDERER: Intel(R) HD Graphics 4600" Debug: "GL_SHADING_LANGUAGE_VERSION: 4.30 - Build 20.19.15.4531"

I see you use the qimage from fbo. It's is slow. You can use the texture id from fbo directly.

[Xuno]

I've made Geometry.h and GeometryRenderer.h public. Usually TexturedGeometry is enough. See OpenGLVideo. I installed your player and tested super xbr, it complains shader error.

Is there a problem with Geometry.h and GeometryRenderer.h ?

[wang-bin]

Is there a problem with Geometry.h and GeometryRenderer.h ?

What problem? I use your installer and XunoPlayer.exe. What's your GL information?

[Xuno]

What's your GL information?

Debug: "OpenGL version: 4.5  hasGLSL: 1"
Debug: "GL_VERSION: 4.5.13464 Compatibility Profile Context 21.19.411.0"
Debug: "GL_VENDOR: ATI Technologies Inc."
Debug: "GL_RENDERER: AMD Radeon R9 200 Series"
Debug: "GL_SHADING_LANGUAGE_VERSION: 4.50"
Debug: "Is OpenGLES: 0"

[Xuno]

We removed use of QImage for first pass and now playback is much smoother.

In what order do shader programs work in QtAV pipe? In afterRendering event we create some passes that generate fbo. But we want to control last stage of texture parameter glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER ... Like here https://github.com/Xuno/Xuno-QtAV/blob/master/examples/player/filters/XunoGlslFilter.cpp#L122

Is this correct?

When its applied before fbo can it be displayed in QtAV ?

[Xuno]

We have a fix coming for Intel HD Graphics, there is a slight difference in code implementation that we use for AMD.

[Xuno]

Intel HD Graphics should work now, please test.

[Xuno]

For now we have implemented adaptive GL filters while Super-XBR testing continues.

[wang-bin]

Is this correct?

Yes

Do you have new installer? I only install a few qt modules and I can't build your player.

[wang-bin]

what is adaptive gl filter?

[Xuno]

Yes, there is a new installer. Please use x1.5 and x2 buttons in the menu for now. We have a problem with filters if the window is dragged larger.

GL filters, sharpness, gamma, etc. values are auto adjusted according to amount of upscaling beyond native frame size. Its works reasonably well with much less overhead.