Shader Preprocessor

[zz85]

I'm opening a suggestion on perhaps how we could do shader preprocessors since manage large chunk of shaders in three.js becomes difficult

Basically it would be some kind of simple macro/template style text replacement

eg.

// Start shader
==COMMON FUNCTIONS==
// Continue shader

basically before compiling, we will just replace ==COMMON FUNCTIONS== with functions mapped by the users, or predefined in the library. Right now we usually do

'//shader start' + ShaderLib['blabla'] + '//shader end'...

I'm thinking a text replacement might be more flexible with text concatenation. For example,

`#DEFINE limits ==LIMITS==

==Limits== can be defined quickly here.

Also, instead of too many #if #endif blocks, the preprocessor might be able to quickly exclude the block if it knows that it would not be used in the shader.

Currently one favorite thing I like doing is

==NOISE== for the noise functions done in the shaders, otherwise copy and pasting huge chunk of code becomes quite unreadable.

As an extra idea, if we have shader file generator in our built process, perhaps our shader code can be stored in plain formatting, instead of a string joined array.

[noiv]

As a user I support this idea. After having successfully started with MeshBasicMaterial an individual look and feel can only be achieved with customized ShaderMaterials. But then time and effort needed raises exponentially and THREE behaves no longer transparent to GL. Isn't a rich, easy to maintain and extend shader library kind of a low hanging fruit to promote THREE and make it distinguishable from other WebGL libraries?

[alteredq]

@zz85 Another rabbit hole warning, here lie dragons. I remember reading somewhere that shader complexity management is one of the biggest practical pain points for game developers. My experience so far concurs.

[noiv]

Here is a shader parser capable of reading text files called TICK: https://dl.dropbox.com/u/354885/tick.js

Usage:

  // makes asynchonous XHR request
  TICK.load("/static/shaders.tick", function(shaders){
    var material = new THREE.ShaderMaterial(shaders.myShaderName);
  });

TICK knows nothing about syntax, qualifiers, structs, etc. it just deals with lines and strings. It can be very picky with whitespaces and multiline comment starting behind code. Also GL ES has features which will make TICK go crazy. But it should be possible to add more understanding of the language step by step. Well, and better error handling, a function to test compilation after load and a syntax highlighter for Sublime. So TICK is no way ready and there no support.

However, a file like this:

'macro muColor
  uniform vec3 color0
  uniform vec3 color1

'macro mucRed
  uniform col3 color (0xff0000)

'shader randomizedLine

  /* 
     creates a red flickering line to be moved at xz plane
  */

  'vertexShader

    varying vec2 posXZ
    void main() {
      posXZ = position.xz
      gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 )
    }

  'fragmentShader

    'mucRed
    uniform float opacity
    varying vec2 posXZ
    float rand(vec2 seed){return fract(sin(dot(seed.xy, vec2(12.9898,78.233))) * 43758.5453);}
    void main() {
      gl_FragColor = vec4( color, rand(posXZ) * opacity )
    }

/* 
   TICK Test 
*/

'shader testBank

  'vertexShader

    uniform float uDisplacementScale (1.0)                // extract THREE uniform with value
    uniform vec2 vUv (2.0, 0.3)
    uniform vec3 directionalLightColor[ MAX_DIR_LIGHTS ]  // do not extract value
    uniform vec4 tangent                                  // extract THREE uniform with default value
    uniform sampler2D tTextureA                           // handle textures
    uniform sampler2D tTextureB                           // handle textures
    void main() {
      /* whatever */
    }

  'fragmentShader

    'muColor                                              // handle macro
    uniform vec4 tangent
    uniform float uDisplacementScale
    void main() {
      // unfinished
    }

is turned into this:

{
  "randomizedLine":{
    "uniforms":{
      "color":{"type":"c","value":{"r":1,"g":0,"b":0}},
      "opacity":{"type":"f","value":0}
    },
    "attributes":{},
    "vertexShader":"
      varying vec2 posXZ;
      void main() {
      posXZ = position.xz;
      gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );
      }",
    "fragmentShader":"
      uniform vec3 color;
      uniform float opacity;
      varying vec2 posXZ;
      float rand(vec2 seed){return fract(sin(dot(seed.xy, vec2(12.9898,78.233))) * 43758.5453);}
      void main() {
      gl_FragColor = vec4( color, rand(posXZ) * opacity );
      }"
    },
  "testBank":{
    "uniforms":{
      "uDisplacementScale":{"type":"f","value":1},
      "vUv":{"type":"v2","value":{"x":2,"y":0.3}},
      "tangent":{"type":"v4","value":{"x":0,"y":0,"z":0,"w":1}},
      "tTextureA":{"type":"t","value":1,"texture":null},
      "tTextureB":{"type":"t","value":2,"texture":null}
    },
    "attributes":{},
    "vertexShader":"
      uniform float uDisplacementScale;
      uniform vec2 vUv;
      uniform vec3 directionalLightColor[ MAX_DIR_LIGHTS ];
      uniform vec4 tangent;
      uniform sampler2D tTextureA;
      uniform sampler2D tTextureB;
      void main() {
      }",
    "fragmentShader":""
  }
}

If possible TICK returns the proper THREE object, below the value is actually a THREE.Vector2, same applies for colors, vec3, etc.

"vUv": {"type":"v2","value":{"x":2,"y":0.3}}

I've added a MIT Licence and if TICK can kickoff a THREE.js shader library, I'd be happy to have contributed.

[zz85]

@noiv interesting.

i was actually thinking more of a simple templating engine, in the simple style of moustache http://mustache.github.com/, for shaders.

the recent shadergraph library also seem to take an approach to simplifying the shader writing process (see http://acko.net/blog/making-mathbox/ and https://github.com/unconed/ShaderGraph.js)

however, templates can be a rather opinionated topic (there are over 1000 templating npm modules) and we have also seen others with different approaches to it eg. http://webglplayground.net/ https://github.com/rdad/PP.js/

i'm going to leave and close this issue for now, since i do not have a good idea on what's best for everyone. i'd love to know if anyone comes out with other elegant solutions though.