Feature Discussion - Shader Common Includes

[ofTheo]

I often find myself pasting the same shader functions at the of my project shaders. i.e. the Photoshop math ones: http://mouaif.wordpress.com/2009/01/05/photoshop-math-with-glsl-shaders/

Something that I thought could really be useful for people working a lot with shaders, is a common shader library for OF.

We would have a folder OF/libs/openFrameworks called shaders/ where we could then have shader code grouped by functionality.

shaders/
   imageProcessing.frag
   blending.frag  ( ie photoshop blend modes ) 
   lighting.frag 
   math.frag  ( shader perlin noise etc )
   computerVision.frag
   dof.frag 
   projectionBlending.frag
   camera.vert
   projectionMapping.frag    

Then to use you would just do:

myShader.include("blending.frag"); 
myShader.include("math.frag"); 
myShader.load("myShader.frag");

Anything that was in blending.frag could then be used by myShader.frag

[ofTheo]

pinging @elliotwoods @arturoc @ofZach

[ofZach]

how would this work in practice (ie, what does myShader.include() do) ? would the app load these shaders from a common place, or do you have to copy them to your data folder?

[ofTheo]

ofShader::include

would copy and paste the contents of the file at the top of your shader ( in memory - not into the actual file ) I guess it would also need to copy it into the data folder so an app could be distributed.

edit: also yes it would initially load it from OF/libs/openFrameworks/shaders/ then afterwards from your data folder.

[arturoc]

this will be useful too for gles2 since OF by default will define some variables for the transformation matrices, colors... so having a file that you can include with a command will make it much cleaner than copying and pasting for every shader

[ofZach]

what if these are in h files w/ stringify http://kile.stravaganza.org/blog/post/glsl_code_stringify -- is there a reason these need to be files?

the flow you are describing (loading first from the common folder, then from data) seems potentially problematic since, for example, we're now through the PG allowing people to create projects at any location. Not sure how a project that's not at the normal directory depth knows how to find the OF root at runtime.

[arturoc]

i'm currently doing that for gles2 the problem is if you want to look at the source for reference you have to go to some .h in OF

but yes finding the folder would be complicated, we could have an OF_ROOT define with the absolute path in the projects but it would need to be in the openframeworks project too

[elliotwoods]

OF_ROOT macro define seems like a decent proposition in itself. back to the point...

i know it seems dangerously close to messing with our parents standards, but couldn't we just parse

#include "libraryShader.frag" //from my data folder
#include <libraryShader.h> //from oF data folder. make a local copy if we don't have it cached locally, e.g. data/shaderHeaders/. complain if unavailable

so we search toUpper(every line of every loaded shader) for #INCLUDE. ignore all white space and grab the filename between a pair of quotes.

if anybody else (e.g. three.js?/others?) are already doing this, then let's follow the same standard. otherwise let's make one that saves time and kills this generation-long-glsl problem

[ofTheo]

I would be in favor of any of these approaches actually :) they each seem to have some advantages / disadvantages

Some notes:

Include approach: If others are using the #include system that could work for us too - at least then its clear in the glsl file that we referencing code from somewhere else. The only downside is we're sort of breaking glsl whereas by doing myShader.include() the glsl files do remain valid.

Copying files to data/: I imagine with any solution where files are copied from a root path, that it can get a little sketchy as people could have non standard setups or be changing the data folder path at runtime. If we go down that route we'll need to do some heavy testing.

Stringify: I agree the stringfy thing is less friendly/flexible but it is nice as its compiles into the app rather then a separate file. However putting all these shaders into .h files somehow feels the most wrong out of a lot of the solutions :)

I think the include solution could potentially be interesting. Its nice as it could also work with people's own shaders ( ie not ones that ship with OF ) also its something that feels so logical that it will be easy to pick up and also quite powerful.

I might take a stab at this and but it in a branch or as a devApp Maybe we could then test it and see how it works in practice?

[bilderbuchi]

Copying files to data/: if we go down that route, could we plan this in a way that we can use it with general assets, not just shaders? thinking of "standard-OF" fonts, graphics, etc here.

[arturoc]

seems you can do something like

pragma include "file.vert"

and the shader reminds valid, not saying that we should go this route, not sure what would be the best.

[arturoc]

and there's also an include extension, although that won't work on gles

http://stackoverflow.com/questions/10754437/how-to-using-the-include-in-glsl-support-arb-shading-language-include

[kylemcdonald]

i think there are two separate suggestions here:

1. make it easy to use things that are "missing" from glsl, like "math.frag" with a noise() and random()
2. make common shader-based techniques more accessible (like DOF, blur, blending)

i'm in favor of doing 1 at the shader level with some kind of "include".

but 2 requires a completely different approach. doing DOF on a mesh means preparing a depth and color FBO, rendering things into them, post-processing the results, etc. doing a blur requires preparing a two FBOs for ping ponging. blending can also require two FBOs depending on how you do it. we shouldn't just have a shader sitting around and expect people to know what to do with it -- instead we should provide classes that implement these "effects" where and give examples that make it clear how to use them.

i think @patriciogonzalezvivo might have a few things to add to this discussion :)

also, in my experience the basic stringify macro does not work for non-trivial shaders due to the variety in preprocessors. in the comments on that post, zach, someone mentions #define STR(...) #__VA_ARGS__ which might work better.

[patriciogonzalezvivo]

As Kyle say, shaders are difficult to standarice because of the technique they need. Some need pingpong some other don't. Personally I like to put them inside code with STRINGIFY because embebed the code and it's more transparent to new users. I think the include could make things even more cryptic and confusing to beginners

I have been breaking and re-arranging some well know shaders and putting them into classes in https://github.com/patriciogonzalezvivo/ofxFX in order to have more flexibility to do pipes between them. But still need lot of love.

Maybe we can think in a better solution if we take a step back and think it as a pipeline obj (ofPipeline) that read take the information of the viewport to allocate the the fbo/pair_of_fbo and then you can "load" a pre-configuration of a pipeline. Internally could be a pointer to core motor of shaders/pingpongs/etc.

For example:

------------------------- setup ofPipeline blur; blur.load(OF_PIPELINE_GAUSSIAN_BLUR);

------------------------ update blur.begin() blur.end()

------------------------ draw blur.draw()

[ofTheo]

@kyle - totally agree that this should just be about 1) I gave example shader categories, but the utility stuff will really be things that aren't cpu + gpu combinations ( like ping-ponging etc )

However things like a recursive blur which use fbo's require the blurring operation at the .frag level. A nice blur function could be part of this utility shader code

[tobiasebsen]

Not long ago I wrote a few shell-scripts to convert shader files into .h files. Might serve as inspiration:

https://github.com/tobiasebsen/ofxOpenVision

There are also a few fragment shaders that might be usefull.

[ofTheo]

thanks @tobiasebsen ! ofxOpenVision looks really interesting, nice collection of shaders too.

[patriciogonzalezvivo]

Wow this is great

[ofTheo]

I've been thinking about this a fair bit. The danger doing anything which copies files when the app is run ( even if it is during the dev process ) is that you're opening up the door to something unexpected happening. I feel like runtime behind the scenes file generation is something to be avoided.

I'm thinking maybe a more manual approach might be better.

Which is we do:

include "math.frag"

but - the include only works if you have the math.frag at the same level as the shader you're including it from.

so essentially this would be the same as what we described above but you would have to copy the shaders you would want to use into you data folder.

I think this could be the most conservative solution and maybe a good place to start. It is also less 'magic' which I think is good as it could be confusing if people didn't know where these shaders were coming from.

If this sounds like an approach worth exploring I can start working on it in a branch for people to review.

Cheers! Theo

[obviousjim]

Hey this sounds good.

Just so it makes sense, the #include is added to the base shader shader, or is this added to the .h file?

On Fri, Dec 14, 2012 at 3:28 PM, Theodore Watson notifications@github.comwrote:

I've been thinking about this a fair bit. The danger doing anything which copies files when the app is run ( even if it is during the dev process ) is that you're opening up the door to something unexpected happening. I feel like runtime behind the scenes file generation is something to be avoided.

I'm thinking maybe a more manual approach might be better.

Which is we do:

include "math.frag"

but - the include only works if you have the math.frag at the same level as the shader you're including it from.

so essentially this would be the same as what we described above but you would have to copy the shaders you would want to use into you data folder.

I think this could be the most conservative solution and maybe a good place to start. It is also less 'magic' which I think is good as it could be confusing if people didn't know where these shaders were coming from.

If this sounds like an approach worth exploring I can start working on it in a branch for people to review.

Cheers! Theo

— Reply to this email directly or view it on GitHubhttps://github.com/openframeworks/openFrameworks/issues/1731#issuecomment-11391055.

• James

[ofTheo]

oh the include would be in the main shader loaded.

ie: myShader.load("", "shader.frag");

shader.frag could have an include to math.frag - which ofShader parses and includes.

[obviousjim]

That's awesome. the one confusion would be may confuse people learning GLSL that #include is a shader language feature, not a part of OF.

Did we consider the possibility of building up the shader through method calls on the object?

shader.addInclude("math.glsl") shader.load("", "shader.frag");

I'm +1 to either way

On Fri, Dec 14, 2012 at 4:29 PM, Theodore Watson notifications@github.comwrote:

oh the include would be in the main shader loaded.

ie: myShader.load("", "shader.frag");

shader.frag could have an include to math.frag - which ofShader parses and includes.

— Reply to this email directly or view it on GitHubhttps://github.com/openframeworks/openFrameworks/issues/1731#issuecomment-11393045.

• James

[ofTheo]

yeah - that was actually my original idea.

but from what I heard other apis are also putting in include ( because its super dumb its not there ).
also at least then when looking at the shader you know that some of the code is in another file.

if we do myShader.include("math.frag") then its only clear in the cpp and seems like magic functions are being called if you look at the glsl file.

I might just give it a try with the #include approach. it could be quite easy then to switch it over if we felt like we didn't want to go that way.

[obviousjim]

Cool, i'm all for it. it will be really useful!

On Fri, Dec 14, 2012 at 4:49 PM, Theodore Watson notifications@github.comwrote:

yeah - that was actually my original idea.

but from what I heard other apis are also putting in include ( because its super dumb its not there ). also at least then when looking at the shader you know that some of the code is in another file.

if we do myShader.include("math.frag") then its only clear in the cpp and seems like magic functions are being called if you look at the glsl file.

I might just give it a try with the #include approach. it could be quite easy then to switch it over if we felt like we didn't want to go that way.

— Reply to this email directly or view it on GitHubhttps://github.com/openframeworks/openFrameworks/issues/1731#issuecomment-11393666.

• James

[bakercp]

I would definitely err on the side of the "non-magic" / transparent approach. This is particularly important for people (even intermediate coders) that are just learning to use shaders by referencing the many shader tutorials online. Most of these tutorials show how to use shaders in a more standard / generic / bare-bones way. I could imagine that it might be difficult to apply what one learns in these tutorials to a more embedded / include-heavy shader system. I don't have a specific suggestion about the best way to do it, but just that we need to keep typical shader learning pathways in mind as we design it. Of course, no matter which way we go, good documentation is key :)

[kylemcdonald]

i don't have a strong opinion about whether it happens in the cpp or in glsl, but i will say if it happens in glsl it should be #pragma include "ofMath.frag" rather than #include "math.frag" for two reasons:

1. using #include means we can't run shaders in standard shader test environments (e.g., OpenGL Shader Builder on OSX) without commenting out lines. #pragma is the right way to do it.
2. ofMath instead of math makes it clear this is an OF feature and not a GLSL feature.

[ofTheo]

good points! agreed. still not 100% sure what is best - but will be good to try it out and see how it feels.

at the very least having a collection of useful shader functions in the release for people to copy and paste into their own shaders would still be very helpful.

[ofTheo]

have a first version of this working. the example test.frag includes two .frags with the #pragma include method.

the regex parsing could use some love in ofShader::parseForIncludes - but it currently seems to work okay.

The branch and example is here: https://github.com/openframeworks/openFrameworks/tree/feature-shader-include/apps/devApps/shaderIncludeDev

It demos doing a simple blur and using the ContrastSaturationBrightness function ( photoshop blending glsl )

Would be great if people could take a look at it.

Cheers! Theo

Here is how the test.frag looks:

#version 120
#extension GL_ARB_texture_rectangle : enable

#pragma include "ofBlendingUtils.frag"
#pragma include "ofImageProcessingUtils.frag"

uniform sampler2DRect tex0;
uniform float blurScale;
uniform float saturation;

void main(){
    vec2 st = gl_TexCoord[0].st;

    //we can grab the original color
    vec4 ogColor = texture2DRect(tex0, st);

    //or we can get a blur 
    //note this is just for testing - a good blur would do it with ping-ponging fbos. 
    vec4 blurColor  =  blurH(tex0, st, blurScale); 
    blurColor       += blurV(tex0, st, blurScale); 
    blurColor *= 0.5; 

    //lets do another blur at a different scale to soften it a bit
    blurColor       += blurV(tex0, st, blurScale*2.2) * 0.7; 
    blurColor       += blurH(tex0, st, blurScale*2.2) * 0.7; 

    blurColor.rgb = ContrastSaturationBrightness(blurColor.rgb, 1.0, saturation, 1.5);

    gl_FragColor = blurColor;
}

and the example / dev app :

[ofTheo]

@patriciogonzalezvivo @kylemcdonald @obviousjim @bakercp curious what you guys think about the above. would be great if you give the https://github.com/openframeworks/openFrameworks/tree/feature-shader-include/ branch a whirl.

my experience was pretty positive. I think the main thing I see is making sure the shader utils collections didn't any inter-dependcy as then that starts to result in glsl code which is less portable.

[kalwalt]

hi to all, just a note: Fragmentarium http://syntopia.github.com/Fragmentarium/ a ray tracing renderer for fractals and other 3Dmath use such a feature for #include different shader. I think you can take a look at a bit for inspiration.

tried the example https://github.com/openframeworks/openFrameworks/tree/feature-shader-include/apps/devApps/shaderIncludeDev but got errors : blurV blurH not defined

[ofTheo]

@kalwalt - oh damn it. I actually managed to not commit my ofShader changes for parsing the #include paths. and since then I reset my branch.

so.... please ignore the branch for now - I'm going to have to rewrite that code. :)

I'll add circular include avoidance to it this time.

thanks for the link to Fragmentarium, thats really useful. I might make it so we support both #pragma include and #include

these lines seem relevant: https://github.com/Syntopia/Fragmentarium/blob/master/Fragmentarium-Source/Fragmentarium/Parser/Preprocessor.cpp#L89

[kalwalt]

Ok , no problem at all! Anyway happy to know that you are working on it . This will be very useful. I was looking in that problem time ago and i found some interesting topic also in the OpenGL forum. If i found again these links i can share these info also.

yes those lines fit exactly what we need!

[chparsons]

Hi there,,

I needed this feature and saw this great thread so I wrote a ofShader::parseForIncludes

It's based on https://www.opengl.org/discussion_boards/showthread.php/169209-include-in-glsl?p=1192415&viewfull=1#post1192415

It's working fine for me with devApps/shaderIncludeDev/ (on osx)

Here's the modified file in my fork of feature-shader-include:

https://github.com/chparsons/openFrameworks/blob/feature-shader-include/libs/openFrameworks/gl/ofShader.cpp#L154

It currently only supports parsing #pragma include for shaders loaded from files.

One thing is that it uses the app/data path to load included shaders, e.g. in cpp: shader.load("", "shaders/test.frag"); in glsl:

pragma include "shaders/ofBlendingUtils.frag"

I wouldn't make the parsing method do anything with the paths in behalf of the user. This would allow him to see and manage the paths clearly the way he wants.

This is related to the issue about paths discussed earlier in this thread.. I think it would be good to have the shaders located per addon, in the addon/data folder or something similar.

But is there a solution for addons assets management with the capability of deploying those assets with each app distribution? I found this issue as well https://github.com/openframeworks/openFrameworks/issues/1599

It ends with a good proposal by @underdoeg: to have "something like copyDataFromAddon(addon, filePath)" That way, each addon would be responsible for managing its assets and could copy them from addon/data to app/data in runtime for app distribution.

In my opinion, the main goal would be to have all external assets, like shaders code or other files, packed per addon..

[kalwalt]

@chparsons i will test on Ubuntu 64bit, it seems very promising!

[kalwalt]

hey @chparsons it works perfectly in Ubuntu 64bit!

[kylemcdonald]

closed by #1795