ptitSeb / gl4es

GL4ES is a OpenGL 2.1/1.5 to GL ES 2.0/1.1 translation library, with support for Pandora, ODroid, OrangePI, CHIP, Raspberry PI, Android, Emscripten and AmigaOS4.
http://ptitseb.github.io/gl4es/
MIT License
694 stars 159 forks source link

gl4es and ScummVM #366

Closed kas1e closed 2 years ago

kas1e commented 2 years ago

Hi ptitSeb !

Today tried to port ScummVM over gl4es, and have some interesting issues: All initialization happens to be fine, and then:

LIBGL: Hardware vendor is A-EON Technology Ltd. Written by Daniel 'Daytona675x' MьЯener @ GoldenCode.eu
LIBGL: GLSL 300 es supported
LIBGL: GLSL 310 es supported and used
Using joystick: Logitech Dual Action
WARNING: GL ERROR: GL_INVALID_VALUE on glCompileShader(handle) (backends/graphics/opengl/shader.cpp:270)!
WARNING: Could not compile shader "attribute vec4 position;
attribute vec2 texCoordIn;
attribute vec4 blendColorIn;

uniform mat4 projection;

varying vec2 texCoord;
varying vec4 blendColor;

void main(void) {
    texCoord    = texCoordIn;
    blendColor  = blendColorIn;
    gl_Position = projection * position;
}
": "ERROR: 24:2: '' : missing #endif 
ERROR: 1 compilation errors.  No code generated.

"!
WARNING: GL ERROR: GL_INVALID_VALUE on glCompileShader(handle) (backends/graphics/opengl/shader.cpp:270)!
WARNING: Could not compile shader "attribute vec4 position;
attribute vec2 texCoordIn;
attribute vec4 blendColorIn;

uniform mat4 projection;

varying vec2 texCoord;
varying vec4 blendColor;

void main(void) {
    texCoord    = texCoordIn;
    blendColor  = blendColorIn;
    gl_Position = projection * position;
}
": "ERROR: 24:2: '' : missing #endif 
ERROR: 1 compilation errors.  No code generated.

"!
WARNING: SearchSet::add: archive 'gui-icons.dat' already present!
Using joystick: Logitech Dual Action 

An interesting moment is that when i build it under MiniGL ( OpenGL 1.3) , it didn't ask for any shaders, and runs fine. But once i just switch to gl4es (without any changes, just as i do all the time), it starts to try to compile some shaders from their internal ScummVM's code.

I think that maybe if they find that OpenGL is anything more than 2.0, then they trying to compile some default base shader, which by some luck didn't work (maybe because they put version 100 or something ? but then, GL4ES should make its own shaders, right ?)

If you have any ideas, feel free plz :) Or at least from where start to debug those things. Thanks!

kas1e commented 2 years ago

Hm, i set to dump all the shaders to log (LIBGL_DBGSHADERCONV to 15), and that is what i have:

#version 110
#ifdef GL_ES
    #if defined(GL_FRAGMENT_PRECISION_HIGH) && GL_FRAGMENT_PRECISION_HIGH == 1
        precision highp float;
    #else
        precision mediump float;
    #endif
#else
    #define highp
    #define mediump
    #define lowp
#endif
attribute vec4 position;
attribute vec2 texCoordIn;
attribute vec4 blendColorIn;

uniform mat4 projection;

varying vec2 texCoord;
varying vec4 blendColor;

void main(void) {
    texCoord    = texCoordIn;
    blendColor  = blendColorIn;
    gl_Position = projection * position;
}

And new shadersource are:

#version 100
precision highp float;
precision highp int;
#ifdef GL_ES
    #if defined(GL_FRAGMENT_PRECISION_HIGH) && GL_FRAGMENT_PRECISION_HIGH == 1
        precision highp float;
    #else
        precision mediump float;
    #endif

attribute vec4 position;
attribute vec2 texCoordIn;
attribute vec4 blendColorIn;

uniform mat4 projection;

varying vec2 texCoord;
varying vec4 blendColor;

void main(void) {
    texCoord    = texCoordIn;
    blendColor  = blendColorIn;
    gl_Position = projection * position;
}

Not sure that i understand what happens, at least i can see that original have all correct ifdefs, then, second shader (dunno by which created ? By gl4es ? ) didn't have all those ifs , so of course fails. Through as i remember all shaders created by gl4es should have gl4es_ marks at beginning of shader's calls, but this one looks strange .. Isn't it gl4es convert the original shader to something before making "gl4es" ones?

ptitSeb commented 2 years ago

Hi @kas1e .

Yes, that looks like a bug in the Shaderconv. In the précompte part to be precise. As a workaround, you can try to use LIBGL_SHADERNOGLES=1 env. var.

kas1e commented 2 years ago

Howdy !:)

Tried workaround: nope didn't help. The same can't compile shader and error about missing ifdef. and white rectangle on-screen.

Btw i also have errors on lining stage about multiple definitions of some ARB functions: glProgramStringARB glBindProgramARB etc.

For all of them says that they were defined n lbgl4es's oldprogram.c. Scummvm code from other side contains that :

#if defined (SDL_BACKEND) && defined(GL_ARB_fragment_program) && !defined(USE_GLEW)

// We need SDL.h for SDL_GL_GetProcAddress.
#include "backends/platform/sdl/sdl-sys.h"

// Extension functions needed for fragment programs.
PFNGLGENPROGRAMSARBPROC glGenProgramsARB;
PFNGLBINDPROGRAMARBPROC glBindProgramARB;
PFNGLPROGRAMSTRINGARBPROC glProgramStringARB;
PFNGLDELETEPROGRAMSARBPROC glDeleteProgramsARB;
PFNGLPROGRAMLOCALPARAMETER4FARBPROC glProgramLocalParameter4fARB;

#endif
kas1e commented 2 years ago

Another moment that if i just do that:

// Taken from: https://en.wikibooks.org/wiki/OpenGL_Programming/Modern_OpenGL_Tutorial_03#OpenGL_ES_2_portability
const char *const g_precisionDefines =
//  "#ifdef GL_ES\n"
//  "\t#if defined(GL_FRAGMENT_PRECISION_HIGH) && GL_FRAGMENT_PRECISION_HIGH == 1\n"
//  "\t\tprecision highp float;\n"
//  "\t#else\n"
//  "\t\tprecision mediump float;\n"
//  "\t#endif\n"
//  "#else\n"
    "\t#define highp\n"
    "\t#define mediump\n"
    "\t#define lowp\n";
//  "#endif\n";

i.e comment out all those ifs/else then while all compiles well but I have a black screen and nothing else. The shaders they use are there:

https://github.com/scummvm/scummvm/blob/master/backends/graphics/opengl/shader.cpp

See there g_defaultVertexShader g_defaultFragmentShader g_lookUpFragmentShader and that g_precisionDefines.

arrowgent commented 2 years ago

renderer set to OpenGL in scummvm options/config renders a white screen when launched or changed and applied

ex: gl4es scummvm where gl4es just _LD_LIBRARYCONFIG to the /lib/libGL.so file provided by compiling gl4es

kas1e commented 2 years ago

@arrowgent Do you mean that happens to you too? On what hardware?

ptitSeb commented 2 years ago

what version of scummvm and gl4es are you using? I tried with latest gl4es using debian version of scummvm and I add no issues, the shaders converted correctly for me.

kas1e commented 2 years ago

I use gl4es-master commit 2484 https://github.com/ptitSeb/gl4es/commit/a05e895d0ea10f513906f18f5418f18bb7baacb9 from 11 Apr 2021

And 5 days old ScummVM build from their auto-build repo.

Will try latest gl4es now

kas1e commented 2 years ago

Tried latest version cant compile:

/amiga/gl4es-master-2567/src/agl/lookup.c:43:80: error: macro "AliasExport" requires 4 arguments, but only 1 given
   43 | void* aglGetProcAddress(const char* name) AliasExport("gl4es_aglGetProcAddress");
      |                                                                                ^
In file included from /amiga/gl4es-master-2567/src/agl/lookup.c:1:
/amiga/gl4es-master-2567/src/agl/../gl/attributes.h:102: note: macro "AliasExport" defined here
  102 |   #define AliasExport(RET,NAME,X,DEF) \
      |
/amiga/gl4es-master-2567/src/agl/lookup.c: In function ‘aglGetProcAddress’:
/amiga/gl4es-master-2567/src/agl/lookup.c:43:43: error: expected declaration specifiers before ‘AliasExport’
   43 | void* aglGetProcAddress(const char* name) AliasExport("gl4es_aglGetProcAddress");
      |                                           ^~~~~~~~~~~
/amiga/gl4es-master-2567/src/agl/lookup.c:44: error: expected ‘{’ at end of input
make[2]: *** [src/CMakeFiles/GL.dir/build.make:1146: src/CMakeFiles/GL.dir/agl/lookup.c.obj] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:114: src/CMakeFiles/GL.dir/all] Error 2
make: *** [Makefile:115: all] Error 2
ptitSeb commented 2 years ago

Ok, this one should be fixed (there may be some more after that)

kas1e commented 2 years ago

At least it fully compiles now thans! I will check scummvm now

kas1e commented 2 years ago

@ptitSeb When tried to ln scummvm have those errors:

/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glProgramStringARB':
oldprogram.c:(.text+0x388): multiple definition of `glProgramStringARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x8): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glProgramStringARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 996 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/li
bgl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glProgramStringARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glBindProgramARB':
oldprogram.c:(.text+0x76c): multiple definition of `glBindProgramARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0xc): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glBindProgramARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 3008 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/lib
gl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glBindProgramARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glDeleteProgramsARB':
oldprogram.c:(.text+0x132c): multiple definition of `glDeleteProgramsARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x4): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glDeleteProgramsARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 452 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/l
ibgl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glDeleteProgramsARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glGenProgramsARB':
oldprogram.c:(.text+0x14f0): multiple definition of `glGenProgramsARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x10): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glGenProgramsARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 2196 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/lib
gl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glGenProgramsARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glProgramLocalParameter4fARB':
oldprogram.c:(.text+0x2404): multiple definition of `glProgramLocalParameter4fARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x0): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glProgramLocalParameter4fARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 280 in /usr/local/amiga/ppc-amigaos/SDK/local/new
lib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glProgramLocalParameter4fARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)

Through they happen to be just for one game so can comment them out.

kas1e commented 2 years ago

@ptitSeb Tested . same problem :( :

WARNING: GL ERROR: GL_INVALID_VALUE on glCompileShader(handle) (backends/graphics/opengl/shader.cpp:269)!
WARNING: Could not compile shader "attribute vec4 position;
attribute vec2 texCoordIn;
attribute vec4 blendColorIn;

uniform mat4 projection;

varying vec2 texCoord;
varying vec4 blendColor;

void main(void) {
    texCoord    = texCoordIn;
    blendColor  = blendColorIn;
    gl_Position = projection * position;
}
": "ERROR: 24:2: '' : missing #endif 
ERROR: 1 compilation errors.  No code generated.

"!
WARNING: GL ERROR: GL_INVALID_VALUE on glCompileShader(handle) (backends/graphics/opengl/shader.cpp:269)!
WARNING: Could not compile shader "attribute vec4 position;
attribute vec2 texCoordIn;
attribute vec4 blendColorIn;

uniform mat4 projection;

varying vec2 texCoord;
varying vec4 blendColor;

void main(void) {
    texCoord    = texCoordIn;
    blendColor  = blendColorIn;
    gl_Position = projection * position;
}
": "ERROR: 24:2: '' : missing #endif 
ERROR: 1 compilation errors.  No code generated.

"!
WARNING: SearchSet::add: archive 'gui-icons.dat' already present!
Using joystick: Logitech Dual Action
LIBGL: Shuting down
LIBGL: Saved a PSA with 1 Precompiled Programs 
kas1e commented 2 years ago

Happens because of that shader https://github.com/scummvm/scummvm/blob/master/backends/graphics/opengl/shader.cpp#L81

kas1e commented 2 years ago

if i short that code to version without ifs/endifs lie:

const char *const g_precisionDefines =

    "#define highp\n"
    "#define mediump\n"
    "#define lowp\n";

} // End of anonymous namespace

Then on running i have that:


WARNNG: GL_ERROR: GL_NVALD_ENUM on glGetProgrammv(_program, 0x8b82, &result) (backends/graphics/opengl/shader.cpp:170) and fail there:

https://github.com/scummvm/scummvm/blob/master/backends/graphics/opengl/shader.cpp#L170
ptitSeb commented 2 years ago

@ptitSeb When tried to ln scummvm have those errors:

/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glProgramStringARB':
oldprogram.c:(.text+0x388): multiple definition of `glProgramStringARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x8): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glProgramStringARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 996 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/li
bgl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glProgramStringARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glBindProgramARB':
oldprogram.c:(.text+0x76c): multiple definition of `glBindProgramARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0xc): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glBindProgramARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 3008 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/lib
gl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glBindProgramARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glDeleteProgramsARB':
oldprogram.c:(.text+0x132c): multiple definition of `glDeleteProgramsARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x4): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glDeleteProgramsARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 452 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/l
ibgl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glDeleteProgramsARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glGenProgramsARB':
oldprogram.c:(.text+0x14f0): multiple definition of `glGenProgramsARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x10): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glGenProgramsARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 2196 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/lib
gl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glGenProgramsARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj): In function `gl4es_glProgramLocalParameter4fARB':
oldprogram.c:(.text+0x2404): multiple definition of `glProgramLocalParameter4fARB'
engines/grim/libgrim.a(gfx_opengl.o):(.sbss+0x0): first defined here
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: size of symbol `glProgramLocalParameter4fARB' changed from 4 in engines/grim/libgrim.a(gfx_opengl.o) to 280 in /usr/local/amiga/ppc-amigaos/SDK/local/new
lib/lib/libgl4es.a(oldprogram.c.obj)
/usr/local/amiga/lib/gcc/ppc-amigaos/8.4.0/../../../../ppc-amigaos/bin/ld: Warning: type of symbol `glProgramLocalParameter4fARB' changed from 1 to 2 in /usr/local/amiga/ppc-amigaos/SDK/local/newlib/lib/libgl4es.a(oldprogram.c.obj)

Through they happen to be just for one game so can comment them out.

Yes, it's a problem because the symbol are exported by gl4es, just in case, and scummvm defined them also (probably some function pointer with the same name). With a dynamic linking, this kind of issue doesn't happens, so yeah, maybe just comment them for now, I don't see any easy fix, instead some mangling of the exported name, but then they will have to be mangled also to use them...

ptitSeb commented 2 years ago

Here is the last shader, using DEBUGSHADERCONV, when launching just the GUI (I haven't setup any game)

Shader source#ifdef GL_ES
        #if defined(GL_FRAGMENT_PRECISION_HIGH) && GL_FRAGMENT_PRECISION_HIGH == 1
                precision highp float;
        #else
                precision mediump float;
        #endif
#else
        #define highp
        #define mediump
        #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D texture;
uniform sampler2D palette;

const float adjustFactor = 255.0 / 256.0 + 1.0 / (2.0 * 256.0);
void main(void) {
        vec4 index = texture2D(texture, texCoord);
        gl_FragColor = blendColor * texture2D(palette, vec2(index.a * adjustFactor, 0.0));
}
:

New Shader source:
#version 100
precision highp float;
precision highp int;
#ifdef GL_ES
        #if defined(GL_FRAGMENT_PRECISION_HIGH) && GL_FRAGMENT_PRECISION_HIGH == 1
                precision highp float;
        #else
                precision mediump float;
        #endif
#else
        #define highp
        #define mediump
        #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D texture;
uniform sampler2D palette;

const float adjustFactor = 255.000 / 256.000 + 1.00000 / (2.00000 * 256.000);
void main(void) {
        vec4 index = texture2D(texture, texCoord);
        gl_FragColor = blendColor * texture2D(palette, vec2(index.a * adjustFactor, 0.00000));
}

As you can see, the original have all the ifdef, and they get converted corretly by shaderconv (using default settings)

ptitSeb commented 2 years ago

And here, using LIBGL_SHADERNOGLES=1

Shader source#ifdef GL_ES
        #if defined(GL_FRAGMENT_PRECISION_HIGH) && GL_FRAGMENT_PRECISION_HIGH == 1
                precision highp float;
        #else
                precision mediump float;
        #endif
#else
        #define highp
        #define mediump
        #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D texture;
uniform sampler2D palette;

const float adjustFactor = 255.0 / 256.0 + 1.0 / (2.0 * 256.0);
void main(void) {
        vec4 index = texture2D(texture, texCoord);
        gl_FragColor = blendColor * texture2D(palette, vec2(index.a * adjustFactor, 0.0));
}
:

New Shader source:
#version 100
precision highp float;
precision highp int;

        #define highp
        #define mediump
        #define lowp

varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D texture;
uniform sampler2D palette;

const float adjustFactor = 255.000 / 256.000 + 1.00000 / (2.00000 * 256.000);
void main(void) {
        vec4 index = texture2D(texture, texCoord);
        gl_FragColor = blendColor * texture2D(palette, vec2(index.a * adjustFactor, 0.00000));
}

Again, it converted properly.

kas1e commented 2 years ago

Hm.. and what can it mean then... Shaderconv's endian or amigaos4 specific issues?

ptitSeb commented 2 years ago

Not sure. Shaderconv should not be really sensitive to endianness, as it only char manipulation.

kas1e commented 2 years ago

now rebuild from scratch everything deleted .gl4es.psa and run again: same error. Lie something weird happens only on my side when shaderconv meet with those "ifs/elses".

ptitSeb commented 2 years ago

this will happens in "preproc.c". Try to put some printf on the #if /#else/#endif handling to see how it goes?

kas1e commented 2 years ago

Aha will try now.

Also what is interesting if modify that shader to not have ifdefs then stll have this error:

WARNNG: GL_ERROR: GL_NVALD_ENUM on glGetProgrammv(_program, 0x8b82, &result) (backends/graphics/opengl/shader.cpp:170)

i tried to compile all shaders manually and they fine.. Somethng weird ..

kas1e commented 2 years ago

@ptitSeb Back today and want to try to deal with that strange issue again :)

So by if/else/endif you mean strings from 464 till 500 in the preproc.c that correct?

What i miss in the new shader after shaderconv is that :

#else
    #define highp
    #define mediump
    #define lowp
#endif

i.e. all after a second else whole buffer part together with else disappear. Lie too small buffer or something .. or some NULL arse from nowhere and split end the buffer in the middle.

ptitSeb commented 2 years ago

from 460 to 588 (it's a state automate).

kas1e commented 2 years ago

@ptitSeb So i back to scummvm tests, and so far while we dealing with shaders and stuff, found another interesting issue. ScummVM lately add "GLAD" to their code, so to not do any "manual function loading", but use GLAD instead. GLAD it just one single header file which manages function pointers for OpenGL , so not manual stuff.

So, there are the file they generated (do not know via command line tool, or just by own GLAD's site there : https://glad.dav1d.de/), but that what they have now in scummvm repo:

https://raw.githubusercontent.com/scummvm/scummvm/21b0d6ded18e6994e43c316759e4ffdb487f244d/graphics/opengl/glad.h

Now, what issue we have now.

If i build ScummVM via GL4ES withut usage of the shaders coming with GL4ES (so only internal ones from gl4es are used), before they add GLAD support, everything renders correctly. See for example Grim Farnando game:

at start:

https://kas1e.mikendezign.com/aos4/scummvm/grim_bug/01_before_glad_grim_start.jpg

in the menu:

https://kas1e.mikendezign.com/aos4/scummvm/grim_bug/02_before_glad_grim_menu.jpg

Now, once they this commit (where they add GLAD instead): https://github.com/scummvm/scummvm/commit/21b0d6ded18e6994e43c316759e4ffdb487f244d

That what i have when i build it with GL4ES:

at start:

https://kas1e.mikendezign.com/aos4/scummvm/grim_bug/01_after_glad_grim_start.jpg

in the menu:

https://kas1e.mikendezign.com/aos4/scummvm/grim_bug/02_after_glad_grim_menu.jpg

See, now there is no main person rendered anymore, and in the menu, we have no sprites at all.

Now, when i compile exactly the same over MiniGL , we do not have this problem. Everything renders correctly and with pre-GLAD addon, and affter GLAD too. I even use exactly the same GL includes from GL4ES for both minigl and gl4es builds. Just once i link scummvm with GL4ES, then after this "GLAD" change i have no main person rendered and sprites missed.

I know, this again will be problem on our side (i am pretty sure it will), but you may at least bring some idea to what to check, etc. Maybe you will notice something "bad" right from the start..

So far i thinking for now about just dumping all the GL4ES internal shaders with pre-GLAD change version, and with-GLAD change version, to compare, if anything changes.

If you have time, you may try also to build latest scummvm sources (those from github ones, not a stable ones) to see how it reacts on Pandora, so we can rule out GL4ES bug/difference. But something make me think it will be bug on our side, again :)

kas1e commented 2 years ago

@ptitSeb I thinking before that maybe there some typo can happens, but generated glad.h is based on specs, so it can't be wrong. Also the fact that almost everything works, but not all, point out that something still wrong with proper function pointer loading. Maybe GLAD do "hook" some function which GL4ES dislike ?

I create ticket on their site : https://bugs.scummvm.org/ticket/13351 , maybe you have anything to add about.

And still interesting moment, is that MiniGL build surely works with GLAD (i use exactly the same includes for both builds, just for MiniGL build i use -lSDL2 -lgl on linking, and for gl4es one -lSDL2_gl42s -lgl4es). So it should't be something about includes per se.

ptitSeb commented 2 years ago

I suppose GLAD find something wrong when it check what extensions are availlable or not, and disable some of it for some reasons. I'll check on the pandora/pyra later.

kas1e commented 2 years ago

Yeah, it should be something like this indeed.. For sake of tests, i tried to build two versions (pre-glad and right-after-glad) and dump internal gl4es shaders (to see, if there will be any difference), and yeah, i can see the difference.

There is one before glad added: https://kas1e.mikendezign.com/aos4/scummvm/grim_shaders_before_glad.txt

There is one after glad added: https://kas1e.mikendezign.com/aos4/scummvm/grim_shaders_after_glad.txt

I can see that shaders changes indeed. And changed very much !

Start checking from line 600.

The just really different, and not only that, but i can see that #extension GL_EXT_frag_depth : enable are added after glad too. But ok, i can understand it only that extensions were now added, but why shaders changes that much ? Seems GLAD really change the way how to compile internal shaders.

kas1e commented 2 years ago

Oh, after carefull look, its not that shaders changes, it that in the second case (after GLAD), we have just addition shaders to be in use with all that "depth_frag" in them.

My bet is that now, after they swith to GLAD, we have enabled GL_EXT_frag_depth , and for that one GL4ES create new shaders, which, fail. I.e. the compiles, but fail to work properly. That can be or simple issue with amigaos4 GL_EXT_frag_depth extension, or in GL4ES (but as always it will be probably something on our side).

At least that how i understand it. Another question why this extension while being enabled and before, only now "enables for real" in scumm vm causing gl4es build new shaders for.

kas1e commented 2 years ago

Oh, now we understand it better.

Problem is usage of the GL_ARB_fragment_program. Before, everything works, because i had to add to GRIM's game code that:

undef GL_ARB_fragment_program

Or it wasn't compiled as cause cross-references with the ARB functions names from gl4es.

So i disable it before, and all works and compiles.

Now, when we swithc to GLAD, all compiles fine, and GL_ARB_fragment_programm start to be in use. And thuse, create new shaders (with that GL_EXT_frag_depth usage), and they just fail by some reassons.

It is expected that when ARB programms in use, then we have those new shaders which tried to use GL_EXT_frag_depth ?

I feel issue can be on our side again, or about ARB extension, or, what is more luckyly GL_EXT_frag_depth usage together with GL_ARB_fragment_program.

ptitSeb commented 2 years ago

Mmm, so more bugs in the ARB program parser I guess...

About the "includes". THose are just regular "GL" and co. include file. Not really from gl4es project. I don't really handle them, and you can switch to some better version if you want.

kas1e commented 2 years ago

Mmm, so more bugs in the ARB program parser I guess...

Not necesasary, as always it can be on our side :)

So far i tred in the shaderconv.c to change int fragdepth = (strstr(pBuffer, "gl_FragDepth"))?1:0; on int fragdepth = 0; so to have shaders create without #extension GL_EXT_frag_depth : enable and now shaders start to looks like this:

#version 100
precision highp float;
precision highp int;
varying mediump vec4 _gl4es_TexCoord_0;
uniform sampler2D _gl4es_Sampler2D_0;

void main() {
    vec4 gl4es_FragDepthTemp = vec4(gl_FragDepthEXT);
    vec4 d;

    d = texture2D(_gl4es_Sampler2D_0, _gl4es_TexCoord_0.xy);
    gl4es_FragDepthTemp = d.xxxx;

    gl_FragDepthEXT = gl4es_FragDepthTemp.z;
} 

And

#version 100
precision highp float;
precision highp int;
varying mediump vec4 _gl4es_TexCoord_0;
uniform vec4 _gl4es_Fragment_ProgramLocal_0;
uniform sampler2D _gl4es_Sampler2D_0;

void main() {
    vec4 level = _gl4es_Fragment_ProgramLocal_0;
    vec4 color;
    vec4 d;
    vec4 sum;

    d = texture2D(_gl4es_Sampler2D_0, _gl4es_TexCoord_0.xy);
    sum = d.xxxx;
    sum = sum + d.yyyy;
    sum = sum + d.zzzz;
    sum = sum * vec4(0.330000, 0.330000, 0.330000, 0.330000);
    sum = sum * level.xxxx;
    gl_FragColor.x = sum.x;
    gl_FragColor.y = sum.y;
    gl_FragColor.z = sum.z;

}

See while first shader without this #extension GL_EXT_frag_depth : enable shader itself stlll here and still with that "FragDepthEXT". Probably to disable FragDepthExt fully i need to disable it somewhere else ?

Just want to check if it extensions cause issues or not.

EDIT: lephilousophe says it's fragdepth because there is MOV result.depth, d.r;\n\ in our fragmentARB, so that probably not related to Exenstion itself .. but .. dunno

ptitSeb commented 2 years ago

The GL_EXT_frag_depth extension is enabled only when neded. The shader must be doing some depth reading. And it's enabled only if the extension is detected at init.

kas1e commented 2 years ago

What i mean, is that i want to try to disable this extension at all from gl4es, to see what shaders will be created then when ARB used, and will it works or not (that can point out us to some direction maybe)

kas1e commented 2 years ago

@ptitSeb I tried to just comment out S("GL_EXT_frag_depth ", fragdepth, 1); in hardtex.c , and LIBGL didn't say now that extension found, and instead i for now have those 2 shaders created after shaderconv:

#version 100
precision highp float;
mediump float fakeFragDepth = 0.0;
precision highp int;
varying mediump vec4 _gl4es_TexCoord_0;
uniform sampler2D _gl4es_Sampler2D_0;

void main() {
    vec4 gl4es_FragDepthTemp = vec4(fakeFragDepth);
    vec4 d;

    d = texture2D(_gl4es_Sampler2D_0, _gl4es_TexCoord_0.xy);
    gl4es_FragDepthTemp = d.xxxx;

    fakeFragDepth = gl4es_FragDepthTemp.z;
}

and

#version 100
precision highp float;
precision highp int;
varying mediump vec4 _gl4es_TexCoord_0;
uniform vec4 _gl4es_Fragment_ProgramLocal_0;
uniform sampler2D _gl4es_Sampler2D_0;

void main() {
    vec4 level = _gl4es_Fragment_ProgramLocal_0;
    vec4 color;
    vec4 d;
    vec4 sum;

    d = texture2D(_gl4es_Sampler2D_0, _gl4es_TexCoord_0.xy);
    sum = d.xxxx;
    sum = sum + d.yyyy;
    sum = sum + d.zzzz;
    sum = sum * vec4(0.330000, 0.330000, 0.330000, 0.330000);
    sum = sum * level.xxxx;
    gl_FragColor.x = sum.x;
    gl_FragColor.y = sum.y;
    gl_FragColor.z = sum.z;

}

And bug still here. So even without usage of GL_EXT_frag_depth but instead with usage of faked one, we still have there such rendering error.

Capehill suggest to try to "hardcore" some values for tests, for example he suggest ti change gl_FragColor.w (alpha channel) to 1.0 in the latter shader as It doesn't seem to be initialized, will check it.

kas1e commented 2 years ago

By the way, can you also plz explain how things works with ARB parser too , is that second shader created by arbgenerator.c ? Because i do search in the whole ScummVM abd GL4ES sources on the sum word, or on vec4 d; and find a shit. Mean that this shader autogenerate somewhere, and it's seems not from ScummVM, but some internal "GL only" generation happens, and then shaderconv made a gl4es version, that correct ?

I just need it to understand where to put gl_FragColor.w = 1.0; inside of GL4ES, so to see if it can be what Capehill suggest or not.

kas1e commented 2 years ago

ps. there is some nitpicking about LIBGL_DBGSHADERCONV - there missing \n in some cases, i.e. phrases looks like Shader source#version 120 (without cariage return after Shadersource words)

kas1e commented 2 years ago

Ok we find out : that shaders come from GRIM's game code of ScummVM: https://github.com/scummvm/scummvm/blob/master/engines/grim/gfx_opengl.cpp#L54

And see, it in some strange format:

static char dimFragSrc[] =
    "!!ARBfp1.0\n\
    PARAM level = program.local[0];\n\
    TEMP color;\n\
    TEMP d;\n\
    TEX d, fragment.texcoord[0], texture[0], 2D;\n\
    TEMP sum;\n\
    MOV sum, d.r;\n\
    ADD sum, sum, d.g;\n\
    ADD sum, sum, d.b;\n\
    MUL sum, sum, 0.33;\n\
    MUL sum, sum, level.x;\n\
    MOV result.color.r, sum;\n\
    MOV result.color.g, sum;\n\
    MOV result.color.b, sum;\n\
    END\n";

I am thinking before, that ARB shaders use GLSL syntax (and in example from SDL2, they are), but there its like some old-school assembler kind mnemonics, and it didn't looks very well like GLSL. So ScummVM send those to GL4ES, and GL4ES translate that "old shader language" to the GLSL shaders , and then at top add shaderconversion for gl4es needs.

Anyway, i add to that shader:

....
MOV result.color.a, 1.0;\n\
END\n";

But it didn't fix rendering at all, same visuall error.

So very much looks like issue with ARB shaders, through unclear where at the moment : ogles2/warp3dnova or gl4es.

ptitSeb commented 2 years ago

It's gl4es that's translate ARB shaders to GLSL shaders. That process may still be a bit buggy sometimes.

ptitSeb commented 2 years ago

Ah, missing the ".a" write on output color? Strange it has to be added, not sure how it's supposed to behave.

kas1e commented 2 years ago

no no, it not fix things, its me tries to add it, because we were in hope that it will fix issues , as .a seems to be unitialized

kas1e commented 2 years ago

I think we there have one of two issues : or ARB shader generation from this old shader format to GLSL convert that shader somehow bad, or, which more luckyly, shader compiled correct, but fail to works on our side by some reassons (through strange why, they so small, and we test much more bigger shaders before and they works)

ptitSeb commented 2 years ago

I still need to build scummvm on the pyra to check how it runs there.

kas1e commented 2 years ago

Yeah, that will rule out (or not) GL4ES at least. Basically those 2 ARB shaders looks simple enough imho, strange why they fail.

But yeah i didn't test before any ARB shaders with GL4ES, i only tested ARB extensions , etc, but they all used GLSL shaders in, not that oldschool ARB ones..

Interesting, if irrlicht engine when use ARB shaders use "real" ARB shaders, and not ARB extensions with GLSL shaders ..

kas1e commented 2 years ago

@ptitSeb How interesting: i build latest ScummVM over my x86/linux. All fine, works fine. Then, i build latest gl4es , and run it like this:

LD_LIBRARY_PATH=/home/kas1e/work/scumm/gl4es-master_2568/lib/ ./scummvm 

And that what i have in output:

LIBGL: Initialising gl4es
LIBGL: v1.1.5 built on Mar 16 2022 10:26:32
LIBGL: Using GLES 2.0 backend
LIBGL: loaded: libGLESv2.so
LIBGL: loaded: libEGL.so
LIBGL: Using GLES 2.0 backend
ATTENTION: default value of option vblank_mode overridden by environment.
LIBGL: Hardware Full NPOT detected and used
LIBGL: Extension GL_EXT_blend_minmax  detected and used
LIBGL: Extension GL_EXT_draw_buffers  detected and used
LIBGL: FBO are in core, and so used
LIBGL: PointSprite are in core, and so used
LIBGL: CubeMap are in core, and so used
LIBGL: BlendColor is in core, and so used
LIBGL: Blend Subtract is in core, and so used
LIBGL: Blend Function and Equation Separation is in core, and so used
LIBGL: Texture Mirrored Repeat is in core, and so used
LIBGL: Extension GL_OES_mapbuffer  detected
LIBGL: Extension GL_OES_element_index_uint  detected and used
LIBGL: Extension GL_OES_packed_depth_stencil  detected and used
LIBGL: Extension GL_OES_depth24  detected and used
LIBGL: Extension GL_OES_rgb8_rgba8  detected and used
LIBGL: Extension GL_EXT_multi_draw_arrays  detected
LIBGL: Extension GL_EXT_texture_format_BGRA8888  detected and used
LIBGL: Extension GL_OES_depth_texture  detected and used
LIBGL: Extension GL_OES_texture_stencil8  detected and used
LIBGL: Extension GL_EXT_texture_rg  detected and used
LIBGL: Extension GL_OES_texture_float  detected and used
LIBGL: Extension GL_OES_texture_half_float  detected and used
LIBGL: Extension GL_EXT_color_buffer_float  detected and used
LIBGL: high precision float in fragment shader available and used
LIBGL: Extension GL_EXT_frag_depth  detected and used
LIBGL: Max vertex attrib: 16
LIBGL: Extension GL_OES_standard_derivatives  detected and used
LIBGL: Max texture size: 16384
LIBGL: Max Varying Vector: 32
LIBGL: Texture Units: 16/16 (hardware: 32), Max lights: 8, Max planes: 6
LIBGL: Extension GL_EXT_texture_filter_anisotropic  detected and used
LIBGL: Max Anisotropic filtering: 16
LIBGL: Max Color Attachments: 8 / Draw buffers: 8
LIBGL: Hardware vendor is Intel Open Source Technology Center
LIBGL: GLSL 300 es supported
LIBGL: GLSL 310 es supported and used
LIBGL: sRGB surface supported
LIBGL: EGLImage from Pixmap supported
LIBGL: EGLImage to Texture2D supported
LIBGL: EGLImage to RenderBuffer supported
LIBGL: Targeting OpenGL 2.1
LIBGL: Not trying to batch small subsequent glDrawXXXX
LIBGL: try to use VBO
LIBGL: Force texture for Attachment color0 on FBO
LIBGL: Hack to trigger a SwapBuffers when a Full Framebuffer Blit on default FBO is done
LIBGL: glX Will try to recycle EGL Surface
LIBGL: Current folder is:/home/kas1e/work/scumm/scummvm-amigaos4
ATTENTION: default value of option vblank_mode overridden by environment.
LIBGL: No EGL configs found (depth=24, stencil=8).
LIBGL: No EGL configs found (depth=24, stencil=8).
LIBGL: No EGL configs found (depth=24, stencil=8).
LIBGL: No EGL configs found (depth=24, stencil=8).
WARNING: Could not compile shader "#version 110
#ifdef GL_ES
    #ifdef GL_FRAGMENT_PRECISION_HIGH
        precision highp float;
    #else
        precision mediump float;
    #endif
#else
    #define highp
    #define mediump
    #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D shaderTexture;

void main(void) {
    gl_FragColor = blendColor * texture2D(shaderTexture, texCoord);
}
": "0:4(1): preprocessor error: Unterminated #if

"!
WARNING: Could not compile shader "#version 110
#ifdef GL_ES
    #ifdef GL_FRAGMENT_PRECISION_HIGH
        precision highp float;
    #else
        precision mediump float;
    #endif
#else
    #define highp
    #define mediump
    #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D shaderTexture;
uniform sampler2D palette;

const float adjustFactor = 255.0 / 256.0 + 1.0 / (2.0 * 256.0);
void main(void) {
    vec4 index = texture2D(shaderTexture, texCoord);
    gl_FragColor = blendColor * texture2D(palette, vec2(index.a * adjustFactor, 0.0));
}
": "0:4(1): preprocessor error: Unterminated #if

"!
WARNING: You are missing a valid 'translations.dat' file. GUI translation will not be available!
WARNING: GUI: Could not find 'gui-icons.dat'!
WARNING: Could not find theme 'scummremastered' falling back to builtin!

And no proper rendering.

That remind me of issue about i say in that topic some posts ago (remember all those missing "ifs" , etc). Feels the same.

Have any clue where to check next ?

But at least it feels like this issue about "unterminated ifs" are coming from GL4ES, just by some luck i didn't have them now on amigaos4, but have on x86 Linux. But i can be wrong of course. But the fact it bring such error on x86/linux rule out amiga which is kind of relief.

After it will works i can check if ARBshaders works or not too.

ptitSeb commented 2 years ago

Yes,that unterminated #if is a gl4es issue. Do you have the converted shader from the example above?

kas1e commented 2 years ago

Yeah, there is it:

Shader source#version 110
#ifdef GL_ES
    #ifdef GL_FRAGMENT_PRECISION_HIGH
        precision highp float;
    #else
        precision mediump float;
    #endif
#else
    #define highp
    #define mediump
    #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D shaderTexture;

void main(void) {
    gl_FragColor = blendColor * texture2D(shaderTexture, texCoord);
}
:

New Shader source:
#version 100
precision highp float;
precision highp int;
#ifdef GL_ES
    #ifdef GL_FRAGMENT_PRECISION_HIGH
        precision highp float;
    #else
        precision mediump float;
    #endif

varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D shaderTexture;

void main(void) {
    gl_FragColor = blendColor * texture2D(shaderTexture, texCoord);
}

WARNING: Could not compile shader "#version 110
#ifdef GL_ES
    #ifdef GL_FRAGMENT_PRECISION_HIGH
        precision highp float;
    #else
        precision mediump float;
    #endif
#else
    #define highp
    #define mediump
    #define lowp
#endif
varying vec2 texCoord;
varying vec4 blendColor;

uniform sampler2D shaderTexture;

void main(void) {
    gl_FragColor = blendColor * texture2D(shaderTexture, texCoord);
}
": "0:4(1): preprocessor error: Unterminated #if

"!

See, its the same 1:1 as i have in previous report on amigaos4 builds : #endif disappeared after shadercov.

kas1e commented 2 years ago

Interesting that on latest/todays amigaos4 builds i didn't have that issue, what make me think it some memory related issue, which appears/disappears in different memory-state-conditions, etc

kas1e commented 2 years ago

And export LIBGL_SHADERNOGLES=1 didn't help on linux the same as before on os4 for that missing endif too.