Support for custom shaders in WebGL

[matthijsgroen]

I'm staring at this for some time now, but have no clue on how to begin in adding it to Crafty.

I would like to add custom shaders to components. It would be awesome if it somehow could follow the same way sprites work.

E.g.

  Crafty.shader('desaturize', 'void native code blabla...'); // I still need to learn how to do this, but there are samples around for the shaders I need

  Crafty.e('spriteName,desaturize').desaturize(0.3, 0.1, 0.4, 0.2); // desaturize, color to desaturize to

Why do I want shaders?

I'm working on a sidescroller that uses parallax scrolling (different layers moving at seperate speeds when the viewport moves). To create a depth effect, I want to images in the background to be desaturized towards the horizon color to create a depth effect. I could do this by changing the images used for the background layers, but this has drawbacks.

Advantages for using shaders for this are:

1. I can use the same images in the foreground and background, by simply scaling the image down and add a desaturation effect. This way I can also use this for entities flying from the background to the foreground and vise-versa
2. When I recolor the background (to simulate a sunrise) the background can be recolored dynamically.
3. Also for the foreground, I could use a sprite, make it darker and blur using shaders

simply said, I can make my game look a lot better, using minimal of sprites (e.g. loading time) and penalty (since the GPU does the calculations)

Is there something like this on the roadmap for Crafty? Any pointers on how I could start with this feature?

[matthijsgroen]

I think this would be a good approach, http://webglfundamentals.org/webgl/lessons/webgl-image-processing-continued.html

Maybe I can create something over the weekend to demo / comment on

[matthijsgroen]

I got a static coded desaturation effect working, next step is to get it dynamic and engine worthy, so that people can add their own custom image filters.

But progress none the less :-) (mostly in my own understanding of webGL and how it is structured within Crafty)

[matthijsgroen]

I got it working for images, but sadly I had to create a ImageWithEffects component as parallel implementation next to the Image component by duplicating its code.

There is no way to override passed shaders and attribute lists.

https://github.com/matthijsgroen/game-play/blob/master/app/scripts/components/shaders/ImageWithEffects.coffee

For sprites, it is even worse. It uses a global Crafty.sprite method that cannot be bypassed. So my implementation is stuck for now :disappointed:

I could make a pull request to fix this, but I'm not sure what the best approach would be.

I tried to create a more generic image filter by using several frame buffers, but my lack of webGL knowledge and the complexity of all abstraction layers within the Crafty implementation made it really difficult.

[matthijsgroen]

@starwed I noticed you did the current WebGL implementation. Would there be a good way to load sprites with a different set of shaders and attributes? (they should have a different implementation of the draw as well to set these attributes) For Image I could make a parallel implementation, but it is not that easy for sprites it seems.

It looks like the first time that Crafty seems to be working against me in what I hope to achieve, and I would like that fixed :smile:

[AndrewRayCode]

ShaderFrog has hundreds of shaders to choose from, and you can edit their sources to see how they work. Here's an example of a desaturation shader applied to an image: http://shaderfrog.com/app/view/116 . You can preview the shader on multiple object types, including a plane, which is probably most applicable for sprites.

[matthijsgroen]

@DelvarWorld The problem is not getting shader code, that is also not really hard to write. The problem is getting the shadercode in Crafty and applied to certain entities.

[matthijsgroen]

What could be a good start would be changing this:

https://github.com/craftyjs/Crafty/blob/78de88eff3d002f1d6a6475bb115621ab15a7060/src/graphics/webgl.js#L158

To accept an extra argument that has a callback that takes an entity.

Basically that callback executes this:

https://github.com/craftyjs/Crafty/blob/78de88eff3d002f1d6a6475bb115621ab15a7060/src/graphics/sprite.js#L243

This way the shader can have all sorts of attributes that can be provided. and will provide them itself given an entity.

next small step would be to have 3 types of default shaders: sprite, image, color the step after that would be developers able to override these shaders

You still have 3 shaders, but at least they can be overridden. Next step would be something per entity or entityclass, but thats a bigger step to take.

This would at least help me setup the shaders I want for what I want to accomplish for my game.

@starwed what do you think? Could I make a pull request for this? would that be merged?

[starwed]

@matthijsgroen

I'd generally be in favor of making it easier to provide custom shaders. I did slightly adjust how they're setup in a PR for layers -- there might a way to make that technique more flexible in a way that would make your job easier.

(BTW, there actually is a wiki page explaining our WebGL implementation, not sure if you've seen that.)

[starwed]

Oh, and I haven't had a chance to look at your proposal in detail yet -- I'll try to find the time this weekend! Definitely open a PR if you've got any work done. (Don't wait til it's "finished", it'll be easier to take into account feedback earlier on. :) )

[matthijsgroen]

I will start on a pull request tomorrow. In the meantime, check out the current shaders I created! http://matthijsgroen.github.io/game-play/dist/index.html

When it it possible on sprites I can change all scenery to sprites, and have enemies realistically moving between the background and foreground

[matthijsgroen]

Fixed by #1003