programmable GL: custom lighting material

[nongio-zz]

ofMaterial implements the lighting models for the light types available in ofLight. Actually to write a custom material shader using the data from ofLightsData() the only way is to replicate the code inside ofMaterial. There isn't an API to easily do that. A use case for custom shader can be a bump map shader or an environment map shader.

[joshuajnoble]

I'd like to see something where shader-ish objects, like ofLight and ofMaterial have a few defined functions that you can put logic in to replace certain functions, i.e. ofLight::pointLight(string pointLightFunction) or ofMaterial::material(string materialFunction) where there's a default that's used in the ofShader if you don't pass those in (still manipulatable via attributes/uniforms).

[nongio-zz]

I think that ofShader should provide the uniforms injection through some sort of templating system, providing lighting data, matrices and maybe some defines to make easy cross platform development eg:

#define SAMPLER sampler2DRect

ofMaterial can use the ofShader utils to build its shader with lighting functions overridable as @joshuajnoble said.

[nongio-zz]

@bilderbuchi isn't this issue section-3d?

[Ahbee]

@arturoc , can ofBaseMaterial handle effects with multiple shader passes?
Wouldn't be better to have a class like ofBaseEffect, which lets users create effects with multiple shaders and passes, but with a single begin() and end().

[arturoc]

@joshua @nongio i've been thinking about being able to inject code somehow into the materials, my main concern is how to do it in a way that doesn't break really bad if we want to change something in the shaders.

One idea is to have some kind of callback system so you could have small functions that do part of the calculation and get parameters for the most common data you need to calculate lighting. The advantage of doing this is that we can know which function is being passed. not sure what those functions should be, right now ofMaterial's shader has pointlight, arealight, directionallight and spotlight, so ofMaterial could have a function that does something like:

material.setPointLightCB(shadersource);

where shadersource is a string with a function like:

void pointLight( in lightData light, in vec3 normal, in vec3 ecPosition3, inout vec3 ambient, inout vec3 diffuse, inout vec3 specular ){
...
}

but we should define what this functions are, probably we can have more fine grained functions than just the type of light?

The other way is to have small bits of includes, that someone can use from their own shaders, i think this is easier to break, if something in those includes changes. with the callback system we can use some kind of smart concatenation to keep old functions syntax working, at least for a while.

@Ahbee are you thinking of deferred rendering or similar? ii guess you can do that already, begin should set the material to the renderer always, so the renderer knows that there's an active material and doesn't try to use the default shaders. then on end you finish the first pass, i guess to an fbo, do the rest of the passes and finally draw

ping @tgfrerer

[Ahbee]

@arturoc , suppose you are doing shadows, which require multiple passes and back face culling . Wouldn't be nice to have an api like this.

void ofApp::draw() {
    shadowEffect.begin();
    mesh1.draw();
    mesh2.draw();
    shadowEffect.end();
}

[tgfrerer]

I would advocate with @arturoc to keep the pair ofMaterial and ofShader as simple as possible, and maybe following a conceptual line similar to Unity or threeJS, if only to stick to the principle of least surprise.

In my mind, a material is a set of parameters that may be used by a shader, which, in code, would eventually manifest as a set of uniforms passed on to the shader. Lights should be independent of material, and should feed their own parameters to the shader, in the same way how i would envisage ofMaterial to work.

Eventually, we would want to upgrade our render-pipeline to use cutting-edge OpenGL, where we this architecture would allow us easily to switch materials parameters to UBOs, therefore providing the GPU with a bind-less accessible library of materials parameters of which it can choose from when shading in bind-less draw calls. I am, by the way @Ahbee, a great fan of your ofxUbo addon!

This would also make it possible to pool all materials parameters data, and to store them tightly packed, which can make a big difference performance-wise (cache misses are amongst the most expensive performance penalties around, even for GPUs).

The Unity documentation is pretty nice, by the way, and the graph they have here (http://docs.unity3d.com/Manual/Materials.html) might explain a little better what i am trying to say.

@ahbee, i'm sure such an API call would be quite nice, but would it be easy to see & learn what was going on behind the scene? And what would be the global OpenGL side-effects of such a shadowEffect.begin() method? (i.e. will there be additional render targets, post-processing layers, etc?) How would these interact with my other, bare-metal OpenGL objects and their state? Which shadow calculation model would it use (stencil volume shadows, projected shadows, etc.)? There's no clear best choice for any of these shadow models, for example, since how best to calculate your shadows highly depends on your scene.

All this leads me to think that such an API would be great for code that builds on top of openFrameworks, like for addons or implementations in projects, but not necessarily for the 3D core.

The danger i could see with including a specialised API is that it could make the graphics core harder to maintain, and slow down core development because more special cases had to be taken into account, implemented & tested.

To sum up: I'm all for making things possible though the core, but would warn against closing out opportunities by making architectural decisions early on in core which should, ultimately, be scene-dependent.

[Ahbee]

@tgfrerer , I was thinking more like an ofBaseEffect class, that can cover all shader effects. Imagine the renderer having a member called currentEffect. Is this not possible?

class ofBaseEffect{
    // the renderer passes all the vbos that called draw() after ofBeginEffect()
    virtual void draw(ofVbo *vbos,int count) = 0;
};

// renderer sets the 'currentEffect' to effect
void ofBeginEffect(ofBaseEffect &effect);

// render calls currentEffect->draw() then sets currentEffect to NULL
void ofEndEffect();

then the user can do something like this

// shadowEffect is subclass of ofBaseEffect
void ofApp::draw() {
    shadowEffect.setParameters(myParams);   
    ofBeginEffect(shadowEffect)
    mesh1.draw(); // renderer wont draw this until ofEndEffect() is called
    mesh2.draw();
    ofEndEffect();
}

If the api was like this couldn't ofMaterial subclass from ofBaseEffect?

class ofBaseEffect{
    void begin(){
        // renderer sets 'currentEffect' to 'this'
    }
     void end(){
       // renderer calls draws on the currentEffect
    }
    // the renderer passes all the vbos that called draw() after ofBeginEffect
    virtual void draw(ofVbo *vbos,int count) = 0;
};

[joshuajnoble]

I'm not clear what exactly would differentiate shadowEffect from a shader would be in this case. A light or a material can be a few different distinct and discrete functions that can be used with easily set parameters. An effect though to my mind at least is a lot more complex, how would you ensure that an effect is applied properly? A shadow is rather complex, as Tim points out, and further effects would either be empty shells where you put a shader (so, in essence, just shaders) or they would be over-limiting and over-complex imho.

On Mon, Sep 8, 2014 at 11:18 AM, Abhi Reddy notifications@github.com wrote:

@tgfrerer https://github.com/tgfrerer , I was thinking more like an ofBaseEffect class, that can cover all shader effects. Imagine the renderer having a member called currentEffect. Is this not possible?

class ofBaseEffect{ // the renderer passes all the vbos that called draw() after ofBeginEffect() virtual void draw(ofVbo *vbos,int count) = 0;}; // renderer sets the 'currentEffect' to effectvoid ofBeginEffect(ofBaseEffect &effect); // render calls currentEffect->draw() then sets currentEffect to NULLvoid ofEndEffect();

then the user can do something like this

// shadowEffect is subclass of ofBaseEffectvoid ofApp::draw() { shadowEffect.setParameters(myParams); shadowEffect.begin(); mesh1.draw(); mesh2.draw(); shadowEffect.end();}

Reply to this email directly or view it on GitHub https://github.com/openframeworks/openFrameworks/issues/3154#issuecomment-54863764 .

joshua noble http://thefactoryfactory.com

[nongio-zz]

@arturoc the callback solution is the most maintainable, anyway I think that probably a simple #include + a flag in the shader class to enable the lightdata upload could solve the issue. @tgfrerer the UBO's solution seems interesting, are UBOs available with the programmable renderer? I have never used them but If I've understood correctly, when using ubos we need to upload ofLightData just one time and then the buffer will be available on all the shaders, Am I right?

[Ahbee]

@joshuajnoble , shadowEffect would require multiple shaders and passes to draw one object. I'm really just throwing out ideas. I think I misunderstand the purpose of ofBaseMaterial. Thanks

@nongio I wrote a ofxUbo class a while back. Right now I have it so each shader gets its own UBO per block(this was the easiest to use API), but I can change it so shaders can share UBOs(I dont know if its more efficient,but it takes slightly more code for the user).

[Ahbee]

@tgfrerer , does it make sense to deprecate the current ofImage::bind() and just create a member called ofTexture diffuseTexture inside ofMaterial to handle default textureing shaders?