GLSL instead of CG

[BielBdeLuna]

With the last commit we have GLSL as the sole language for the engine, currently we generate the GLSL files from the original CG files the engine came with, once generated the GLSL files, the CG files are no longer needed.

This method was already used by doom3BFG where GLSL files and HLSL files where being generated from CG files (HLSL and CG being both created in part by Microsoft, are very similar) HLSL files though are not used, nor CG, when in-game only GLSL files are used, just at the start of the engine, the CG was converted to GLSL.

HLSL was used somehow in the xbox 360, CG in the PS3, but none of the code remains in the engine, and GLSL was used on Windows, Mac, and Linux.

so why not stick with CG? because the conversion is a wasteful process, and because CG will no longer be updated as it's been deprecated by Nvidia (their sole maintainer), and because, if needed, code could still be translated back to CG or HLSL, with some added effort.

Why was CG wasteful: during the translation of CG code to GLSL, only the types where changed, from Float4 to vec4 and stuff like that, so any variable "float4 colour" in the CG file was translated as "vec4 colour" into GLSL, and like this with any other type that needed changes where translated accordingly. also some variables changed name following a convention stated in the c++ code.

But between languages there was also changes in the base functions, like the one the extract the colour of a pixel from a given image, in CG it reads like this: float4 tex2D(sampler2D samp, float2 s) page 774 on CG documentation in GLSL it became something like this: gvec4 texture (gsampler2D sampler, vec2 P [, float bias] ) page 98 on GLSL documentation so the code is written for a function named "tex2D" that is the same as the "texture" function that takes the same parameters. then, the clever guys at ID came up with a good solution, they added to every GLSL a part of code with some specific functions, like in the case of every fragment shader, the following would be added:

    "#version 100\n"
    "#define GLES2\n"
    "#define PC\n"
    "precision mediump float;\n"
    "#extension GL_OES_standard_derivatives : enable\n"
    "\n"
    "void clip( float v ) { if ( v < 0.0 ) { discard; } }\n"
    "void clip( vec2 v ) { if ( any( lessThan( v, vec2( 0.0 ) ) ) ) { discard; } }\n"
    "void clip( vec3 v ) { if ( any( lessThan( v, vec3( 0.0 ) ) ) ) { discard; } }\n"
    "void clip( vec4 v ) { if ( any( lessThan( v, vec4( 0.0 ) ) ) ) { discard; } }\n"
    "\n"
    "float saturate( float v ) { return clamp( v, 0.0, 1.0 ); }\n"
    "vec2 saturate( vec2 v ) { return clamp( v, 0.0, 1.0 ); }\n"
    "vec3 saturate( vec3 v ) { return clamp( v, 0.0, 1.0 ); }\n"
    "vec4 saturate( vec4 v ) { return clamp( v, 0.0, 1.0 ); }\n"
    "\n"
    "vec4 tex2D( sampler2D sampler, vec2 texcoord ) { return texture2D( sampler, texcoord.xy ); }\n"
    //"vec4 tex2D( sampler2DShadow sampler, vec3 texcoord ) { return vec4( texture( sampler, texcoord.xyz ) ); }\n"
    "\n"
    //"vec4 tex2D( sampler2D sampler, vec2 texcoord, vec2 dx, vec2 dy ) { return textureGrad( sampler, texcoord.xy, dx, dy ); }\n"
    //"vec4 tex2D( sampler2DShadow sampler, vec3 texcoord, vec2 dx, vec2 dy ) { return vec4( textureGrad( sampler, texcoord.xyz, dx, dy ) ); }\n"
    //"\n"
    "vec4 texCUBE( samplerCube sampler, vec3 texcoord ) { return textureCube( sampler, texcoord.xyz ); }\n"
    //"vec4 texCUBE( samplerCubeShadow sampler, vec4 texcoord ) { return vec4( textureCube( sampler, texcoord.xyzw ) ); }\n"
    "\n"
    //"vec4 tex1Dproj( sampler1D sampler, vec2 texcoord ) { return textureProj( sampler, texcoord ); }\n"
    "vec4 tex2Dproj( sampler2D sampler, vec3 texcoord ) { return texture2DProj( sampler, texcoord ); }\n"
    //"vec4 tex3Dproj( sampler3D sampler, vec4 texcoord ) { return textureProj( sampler, texcoord ); }\n"
    "\n"
    //"vec4 tex1Dbias( sampler1D sampler, vec4 texcoord ) { return texture( sampler, texcoord.x, texcoord.w ); }\n"
    //"vec4 tex2Dbias( sampler2D sampler, vec4 texcoord ) { return texture( sampler, texcoord.xy, texcoord.w ); }\n"
    //"vec4 tex3Dbias( sampler3D sampler, vec4 texcoord ) { return texture( sampler, texcoord.xyz, texcoord.w ); }\n"
    //"vec4 texCUBEbias( samplerCube sampler, vec4 texcoord ) { return texture( sampler, texcoord.xyz, texcoord.w ); }\n"
    //"\n"
    //"vec4 tex1Dlod( sampler1D sampler, vec4 texcoord ) { return textureLod( sampler, texcoord.x, texcoord.w ); }\n"
    //"vec4 tex2Dlod( sampler2D sampler, vec4 texcoord ) { return textureLod( sampler, texcoord.xy, texcoord.w ); }\n"
    //"vec4 tex3Dlod( sampler3D sampler, vec4 texcoord ) { return textureLod( sampler, texcoord.xyz, texcoord.w ); }\n"
    //"vec4 texCUBElod( samplerCube sampler, vec4 texcoord ) { return textureLod( sampler, texcoord.xyz, texcoord.w ); }\n"
    //"\n"

As you can see "tex2D" is there as well as "texture" in the same line, so since in GLSL there isn't a function named "tex2D" they've created one so they don't have to parse the CG files, the rest of the file is the CG file minus the comments. so all this list is all the different functions the CG used in all fragment shaders converted to GLSL, even if you don't use any of them in your little fragment shader CG file, you'd get all this stuff copied in your GLSL translation

And ultimately there is another reason: the GLSL spec for 3.30 version is only 116 pages long while the latest CG spec for CG 3.1 is only 1027 pages long (as it covers the CG API, the CG API for OpenGL, for DX9, for DX10, for DX11 etc... ) so the documentation is compacter, more to the point, and with relative ease of understanding, I recommend to download the GLSL documentation: https://www.khronos.org/registry/OpenGL/specs/gl/GLSLangSpec.3.30.pdf

so now with my PR this can end:

now all the CG code is in it's own directory named /renderprogs/cg/ so rendreprogs is free to contain just GLSL code

as I explained in my pr, right now there is still CG conversion going on, the engine looks for CG files, and if they exist and the engine is forced (which currently is forced by default, until we supply GLSL files ourselves ) then the engine converts the CG files from their folder to the appropriate GLSL files in the /renderprogs/ directory, as well as making a copy of those converted files in their appropriated GLSL directory, as it did before (but now it uses the GLSL files in the /renderprogs/ directory ) the engine can use several different version of GLSL depending on what GPU you have in your machine, and on the level of the GPU you have there, as well as the drivers you use. The engine currently it uses OpenGL32 core profile which uses the GLSL version 1.50, although the lowest machine that runs the engine at OpenGL32 core profile (yeah, my machine XD ) is capable of OpenGL3.3 at core profile, but this stuff is for future pull requests.

there is a new cvar with my PR: idCVar r_sourceGLSL( "r_sourceGLSL", "0", CVAR_BOOL, "Whether if we use GLSL files as source for the shaders or CG files");

as this piece of code states currently is defaulted off if we want to use pre-existing GLSL this has to be turned on when starting the engine adding the following command after the engine

+set r_sourceGLSL 1

but first we have to create all the GLSL files to start playing with them! for this we have r_alwaysExportGLSL which is defaulted to 1 so the engine works out of the box with the CG code.

and here comes the only bug I couldn't solve:

if we set r_sourceGLSL to positive and then set r_alwaysExportGLSL to negative, not all CG files get translated

so if we're working on the GLSL files there is risk of overwritting your work with the "crap" in the CG files! so right now as a solution I propose the following: 1) start the engine with +set r_sourceGLSL 1 +set r_alwaysExportGLSL 1 2) and once completed the conversion quit and re-start the engine but this time with +set r_sourceGLSL 1 +set r_alwaysExportGLSL 0, or even better change r_alwaysExportGLSL to 0 as default behaviour in the c++ code.

why not distribute already GLSL files? instead of keeping redistributing the CG files? because the GLSL as they are created now, they don't contain any comment, and have excess code that should be eventually cleaned, and also there are some things we could do that Motorsep does in his engine fork, that is heavy use of the "import" statement, so GLSL files (that most of them are copies of one another with different names) could be very small.

once we have cool GLSL code we could stop distributing the old CG files and distribute our own GLSL code as well as invert the default value of r_sourceGLSL and r_alwaysExportGLSL in c++ code.