Provide a way to add custom GLSL functions in JS API.

[nsthorat]

From @znah on September 11, 2017 12:9

One can also think of integration with SaderGraph to support users code to make queries to Tensors while abstracting away the data texture layout details. This way we can implement have MathBox-like functionality that uses deeplearn.js under the hood.

Copied from original issue: tensorflow/tfjs-core#121

[dcunited001]

shadergraph doesn't support webgl2 and can't be easily modified to do that. its internals parse GLSL code into an AST, but the parser is locked to webgl 1.0 and can't easily be modified.

does TFJS use webgl 2?

[dcunited001]

it looks like TFJS does pull a webgl2 context, if available. but there aren't any constants associated with webgl2-specific features in the shader metaprogramming in tfjs-core.

@nsthorat if you want to do this, i would recommend running TFJS while Webgl-Inspector is running, so you can ID the textures it creates. if it seems feasible, you can compile your own shaders and push them into storage outside TF on each frame or compute cycle. however, getting the metaprogramming from both Shadergraph and TF play well together will be tough.

however, since the TF textures represent tensors and bc TFJS doesn't really use WebGL, the data in those textures will be heavily deconstructed. WebGL 1.0's feature set has heavy design constraints for shader programming.

[nsthorat]

We are going to expose this after we release, relevant methods were exported in this PR (thanks @aman-tiwari) https://github.com/tensorflow/tfjs-core/pull/1202

We'll add a tutorial soon on how to do this.

[dsmilkov]

Should we close this issue, or keep to track the tutorial? @nsthorat @aman-tiwari

[nsthorat]

Let's keep it open for the tutorial then we can close it

[aman-tiwari]

Tutorial at https://github.com/tensorflow/tfjs-website/pull/186

[nsthorat]

This is done.