add getTexture for Effect

[lselden]

Feedback effects can be created pretty easily if the "getTexture" method of the Target class is duplicated by the Effect class:

this.getTexture = function () {
  if (me && me.frameBuffer) {
    return me.frameBuffer.texture;
  }
};

Source already allows WebGLTexture as an input type, so the result can be used as any other source node.

The result is pretty useful - I've created a game of life shader that uses this method, and you can also take a look at a video feedback effect here: http://www.lukeselden.com/vj/

[brianchirls]

First of all...awesome. Your VJ demo made my day. I'd love to see the game of life shader.

I don't see any reason not to do this, so sure, okay.

Right now, Seriously doesn't allow loops in the network - it's a Directed Acyclic Graph. But loosening this restriction may be a more elegant way to do what you're talking about, as long as the code discovers any cycles and references a the previous frame rather than lock up the browser in an infinite loop - kind of like how puredata does it.

Thoughts, anyone?

[forresto]

Meemoo allows loops[1] everywhere because each module holds its own canvas. I want to go more in Seriously's direction for the next version... where do you do the graph analysis?

[1] 2D canvas, Firefox blend mode: http://meemoo.org/iframework/#gist/5474898

[brianchirls]

Something to be aware of - the texture won't be available until after the node is rendered the first time. We may want to revisit how that works if this is something people want regularly. Also, remember Seriously can't create any textures until it has a canvas attached, because it needs a canvas to create a WebGL context.

[brianchirls]

@forresto Graph analysis happens here: https://github.com/brianchirls/Seriously.js/blob/develop/seriously.js#L2059

[lselden]

What I'd like to see is a wrapper around FrameBuffer that copies the WebGLTexture of whatever's set as its source. This could be done as a final task at the end of each render (either globally, or after the draw function of the source node). The downside is that it would require an additional Texture in memory. The upside is that updating the texture could be turned on and off, which would be useful for a number of tasks, such as:

• Background subtraction (capture frame, keep it)
• Stutter frame effects (capture every 3rd frame)
• Or instantaneous purposes (video feedback, motion detection)

Re texture availability: It looks like TargetNode takes a callback as an argument that can be run after the node is first drawn -- that's what I used to start the feedback loop. However, it didn't work consistently so I had to add a deferred check:

// setup
var s = Seriously();

var input = s.source(); // some input
var transform = s.effect('lumakey'); // some transform effect
var mixer = s.effect('blend');

transform.source = input; // will replace with feedback loop 
mixer.top = input;
mixer.bottom = transform; 

// get texture
var startFeedback = function () {
    var buffer = mixer.getTexture();
    if (!buffer) {
        setTimeout(startFeedback, 1000);
    } else {
        transform.source = buffer; // use texture = feedback fun
    }
}

[lselden]

Also, here's my game of life effect (and a dither effect somewhat inspired by Meemoo):

http://www.lukeselden.com/vj/gameoflife.html?w=1280&h=720

http://www.lukeselden.com/vj/src/gameoflife.js

[brianchirls]

I've thought more about this, and I don't think getTexture is the right way to go. The thing is that I'm thinking about a new performance feature for which I want to be able to swap around the texture for a given node at any time. Many effects have some parameter(s) that, at certain values, can cause the effect to do nothing. For example, a split that's set to show 100% of one side, or a blur with a radius of 0. Instead of running through the whole calculator, or even copying the texture, I want to allow an effect's draw method to simply point to the texture of one if its input nodes in cases where it doesn't have to do anything. In that case, getTexture could point to a different WebGLTexture every time, making it impossible to use a Source node on that texture.

I think a better approach is to build an effect similar to the "Accumulator" in Quartz Composer: http://quartzcomposer.com/patches/366-accumulator

I could even add a blend mode option as well as an opacity amount to determine the quality and degree of each accumulation. There would also be boolean options for whether to reset the buffer and to turn accumulation on and off. That way, you could achieve your stutter effect by turning accumulation on every X frames and then back off again.

Would this suit your needs?

[lselden]

Accumulator is exactly what I'd need! ...as long as it allows loops in the render graph.

[brianchirls]

I'll see what I can do.

I'm gonna pull getTexture out of the develop branch and close this issue. Will open up a new one for accumulator.