2d canvas / webgl

[forresto]

Thinking about React.js treating the browser more like GPU / game engine programming ("repaint whole app, only changed parts get wet").

The SVG #66 branch is going to make our current design nicer to implement.

But planning for the future, I think we're going to end up doing everything in WebGL for the control and speed. How could React's concept of virtual DOM be structured with FBP logic and WebGL views?

Cons:

• lack of css (?)

[forresto]

Since the structure is built on an immediate mode metaphor, and renderers are just functions, it should be possible to build on the react/svg work to move to a canvas/webgl game engine when speed or aesthetics makes it attractive:

• https://github.com/facebook/react-art -- svg and canvas, battle-tested with react
• https://github.com/ericdrowell/KineticJS/wiki -- canvas, nice api
• http://lib.ivank.net/?p=demos&d=bezier -- webgl, already has curves
• https://github.com/GoodBoyDigital/pixi.js/ -- webgl and canvas
• http://fabricjs.com/ -- canvas
• http://www.createjs.com/#!/EaselJS/demos -- looks like Flash
• http://paperjs.org/examples/hit-testing/ -- battle-worn scene graph, vector focus, 2d canvas output
• https://github.com/piqnt/cutjs - simple scene graph

The main thing to test in canvas/webgl engines is if their hit-testing can differentiate overlapping curves. When I checked at least pixi could only hit-test rectangles and ellipses.

Paper.js looks like the winner for our use case. :apple:

[forresto]

They slipped hit-testing into the standard canvas 2d when I wasn't looking, but it isn't implemented yet :trollface:

This hack might work for 2d: https://shifteleven.com/articles/2012/01/08/mouse-clicks-for-html5-cavas-entities-via-paths/

[subtleGradient]

The trick I've seen used with canvas is to draw the same objects with a solid aliased color into a separate canvas. Then when you click, get the pixel color on the hidden canvas and match it up to the real object using a lookup table of pixel color keys to real object values. Not sure if that's the same technique that shift eleven is using or not.

[forresto]

Looks like a different technique, but seems cleaner, and faster. As long as we have < 16777216 edges :wink:

Shift Eleven is keeping track of all paths and then writing them in reverse order on every click. With 10000 objects you start to notice.

:bulb: Thanks!

[forresto]

idea

While animating (zoom, pan, drag node) switch to a 2d canvas view for 60fps goodness. Then release to rerender SVG.

Then we still get the text and click advantages of SVG. A simplified canvas renderer wouldn't need all the bells and whistles. Just general shapes and colors, a little more than the-graph-thumb. Canvas' setTransform() matrix should be enough to keep things aligned with the SVG's transform matrix.

Would need tests to see how jarring switching between would be.

[subtleGradient]

Maybe always render using canvas with SVG layered on top for text and noodles. Then hide the SVG while zooming and panning.

[forresto]

This will help: https://hacks.mozilla.org/2014/03/introducing-the-canvas-debugger-in-firefox-developer-tools/

[forresto]

WebGL implementation of SVG, focused on d3: http://pathgl.com/ (Missing text and curves, which is most of what we have. And there is still no WebGL on iOS.)

[forresto]

Pixi/webgl drawing of force-directed graphs https://github.com/anvaka/ngraph/tree/master/examples/pixi.js

Bezier demo: http://jabtunes.com/labs/3d/bezierlights/

[forresto]

Tiling techniques: http://chairnerd.seatgeek.com/high-performance-map-interactions-using-html5-canvas/

[subtleGradient]

MathBox2 looks almost ludicrously fantastic: http://acko.net/files/pres/siggraph-2014-bof/online.html

[subtleGradient]

iOS 8 has WebGL enabled for Safari and WebViews by default! http://caniuse.com/#feat=webgl

[subtleGradient]

ShaderGraph 1 could link up snippets of GLSL into pipelines, by matching type signatures.

You could construct arbitrary directed acyclic graphs out of snippets, which got compiled into a single vertex and fragment shader. It's driven by code though, it's not a graphical UI.

ShaderGraph 2 fixes some bad design decisions, and adds one major change: callbacks. It turns the traditional data flow into something functional, allowing e.g. the getArrowGeometry shader to call getSample as much as it wants. The data flow is essentially redirected to a completely different part of the graph.

But callbacks can be graphs themselves. Any open inputs or outputs are bundled up into its type signature. This is what allows complicated behavior to be wrapped up and re-used in a completely modular fashion.

[forresto]

Yeah, I emailed with @unconed about making a flowhub runtime for ShaderGraph 2. Looking forward to playing with it.

[jefffriesen]

These look interesting: https://github.com/Izzimach/react-three https://github.com/Izzimach/react-pixi

[jonnor]

Not gonna happen in this codebase