Export Feature (THREE.js)

[Bug-Reaper]

Wondering if I can put in some work here to buff out a three.js flavored export. For re-usable ShaderMaterials, I usually have them as their own little factory function like so:

// File named - ShaderNameHere.js
import * as THREE from 'three';

export function ShaderNameHere() {
    this.uniforms = {
        iTime: { value: 70.0 },
        iMouse: { value: [0, 0] },
        iResolution: { value: [2000.0, 1000.0] },
    }
    this.render = (delta) => { this.uniforms.iTime.value += delta}
    this.material = new THREE.ShaderMaterial({
        uniforms: this.uniforms,
        side: THREE.BackSide,
        vertexShader:`...`,
        fragmentShader:`...`
    })
}

Then to use the custom shader material, you can:

import * as THREE from 'three';
import { ShaderNameHere } from './ShaderNameHere.js'

const shaderController = new ShaderNameHere();
const geometry = new THREE.SphereGeometry()
const mesh = new THREE.Mesh(geometry, shaderController.material)

/*
... Then somewhere in your render loop
*/
shaderController.render(delta) 

---

Wanted to see how you feel about the three.js flavored export to be a .js file similar to the first snippet?

Think I could bang out a PR for the exporter and have some good ideas for handling stuff like textures etc... Also happy to throw together a more robust export usage explanation page as part of the PR.

[AndrewRayCode]

Yes, Shaderfrog needs a first class export feature for all of the engines, including three. What you have posted is an option for exporting if there is no Three.js material/lighting in the graph.

Shaderfrog 1.0 Exports

In the first version of Shaderfrog, the export was a JSON file, like this which encodes the GLSL of the shader, and the runtime material parameters. Having the material as a data file is good for interoperability, because multiple editors can agree on the format for things like uniforms, nodes, ranges, etc.

With the above JSON file, the expectation is you import the shaderfrog runtime library, which parses the JSON and creates the material.

Shaderfrog 2.0 Exports - How Shaderfrog works

There's an issue with Shaderfrog 2.0 and static data files. Shaderfrog supports engine material shaders, like THREE.MeshPhysicalMaterial, with a runtime compilation step. For a shader like this one, you can see there's a "MeshPhysicalMaterial" node in the graph, which is the standard Three material. The way this works under the hood is important:

1. When Shaderfrog compiles this graph, it creates a temporary/throwaway MeshPhysicalMaterial
2. It then forces a compilation of the temporary material, and captures its full GLSL source code. Said another way, Shaderfrog scrapes the generated GLSL from the Three material
3. Shaderfrog parses the Three material GLSL into an AST (this in-memory AST is part of the "context" of the graph, the AST only exists at runtime)
4. When Shaderfrog compiles the graph into the final material (in your code block, final material = vertexShader / fragmentShader), it manipulates the Three AST to compose it with other nodes in the graph. It's conceptually similar to doing string find/replace in Three's onBeforeCompile on materials, but it's more robust since it knows the names of all variables, uniforms, etc, and can do things like make sure variable names don't conflict.

The critical aspect of this process is step 2. Three.js generates unique shader source code based on the available features of the device/browser, and based on what's in the scene. For example the Three.js built in material source code changes based on the number of lights in your scene. That's why the Shaderfrog editor forces a scene render to compile shaders, and does a runtime build of the shader, since the final source code is dependent on the context the shader is used in. Babylon and Playcanvas work the same way.

Said another way, if your graph contains a Three standard material, the final export file probably can't be as simple as simply including the final GLSL from what was compiled on your machine in the editor. This material GLSL won't line up if someone imports the same file into their scene with different lights or different device features (webgl1 vs webgl2 support for example).

Possible solution - precompilation?

One option is to precompile all the variations of the shader you know your scene will need. Like if you know your scene will have up to 2 spotlights and up to 2 pointlights, you could precompile all possible 4 variations of that scene setup, and have all 4 shaders lying around that you swap out based on your scene configuration. The Three.js community aggressively hates compile steps though, so I don't think precompilation will catch on. Also for engines like Babylon, their shaders are hyper-dynamic, so precompiling shader variations probably isn't feasible since there would be too many combinations.

If your shader has built-in three.js materials in it, and your shader composes in other shader nodes, I think there has to be some kind of runtime compilation step, since you can't export the static GLSL for a three material (above), you have to let Three.js generate it, then hijack it and compose in your own shader nodes at runtime (which is the whole premise of the shaderfrog editor).

Exporting option: API usage

I have started an incomplete example about one possible way to do this: https://github.com/ShaderFrog/examples/tree/main/three

This example is what a developer might do if they're building their own their own engine, and not using the Shaderfrog editor. They could use the Shaderfrog core API directly. It mirrors the editor graph. For example, you can create graph nodes using the API:

const physicalF = engine.constructors.physical(
...
);

Including nodes with source code:

  core.sourceNode(
    id,
    'Julia',
     `precision highp float;
      precision highp int;

      uniform vec2 start;
      uniform int iter;`
...

And then you do a runtime compilation of your graph and shove that whole runtime generated output into a Three.js material: https://github.com/ShaderFrog/examples/blob/main/three/index.js#L222

There's a codesandbox demonstrating this linked from this repo as well

This has been a long winded way to say: one option for exporting is to basically generate the code linked above. Generate a runtime JS file (like you have) except that it uses the Shaderfrog core API functions (and compiler) to build the material at runtime, rather than the fully static shader source code file you posted. This has some implications:

1. For this to work, your runtime code needs the shaderfrog API + compiler source code, basically your code needs these imports. This isn't a huge amount of code but it is a few extra 10s of kb that you need in your own source code.
2. That has the obvious implication that you can't export a completely standalone Javascript file like you have, to use that Javascript file you'd need to import the Shaderfrog libraries.
3. A bigger issue here is Shaderfrog compilation is slow, because it's doing a full parse/compile of source code in Javascript. In the editor, the compiler is pretty unoptimized. Compilation takes seconds, which is of course too long for someone editing + live reloading their scene while developing, and too slow for a final end user's device to do when playing the game / whatever.

For point 3, I'm confident I can get the compile time down below a second. And there's also likely a bunch of tricks / shortcuts to make this more efficient. For example, an optimization is probably to do simple string replacement on the Three.js shader source code instead of fully parsing it. Individual shader nodes in a graph might still need full parsing (maybe).

Caveat: What if there's no Three material in the graph?

The very first thing I mentioned in this reply is that your source code file is an option if there is no Three.js material in the graph. Why? Well, because end user shaders don't usually rely on Three.js lights/etc. If you want a material that doesn't use any of Three.js's features, like lighting, refraction, vertex animations, morph targets, etc, then yes, you probably could export just a static shader source code file like you have.

There's still a question here. What if your exported material is the result of composing multiple graph nodes? What should the exported file contain? Should it be the final compiled GLSL, all mashed together into one big string? A downside of this is you can no longer trivially modify effects in isolation. For example this neato box shader is composed of lots of individual shader nodes. If you export this for use in your game, with the file you have now, it would be the compiled GLSL all mashed together. (Go to the Shader -> Fragment subtabs in that link to see what it looks like). Vs, if you open an individual node in that graph, like "Tunnel Perlin Noise" - you can modify effect code in isolation.

Maybe it's ok that the final exported shader is mashed together. That means the workflow for a user would be:

1. Make your shader in the shaderfrog editor, not in your engine
2. Export the shader for use in your engine, and use uniforms to tweak the shader at runtime. By the way a neat feature I haven't yet added is the ability to "bake" in a uniform, aka hard code it, so that you can reduce the number of uniforms in the exported shader to only be the ones you know will change at runtime. This lets you make the shader hyper configurable in the editor with many uniforms, but lock it down to only the ones you need to change dynamically in your game/export.
3. If you want to modify the shader, you basically have to do it back in the editor and re-export. Or you could modify the compiled GLSL, it's just more annoying.

Putting it all together - options for exporting

Understanding the range of features the editor supports is important when considering the export format for shaders. Here's the options as I see them:

1. Export a static JSON file, like shaderfrog 1.0, that is parsed and compiled at runtime
2. Export a runtime JS file like you have, that is parsed and compiled at runtime, where most of the data is static/JSON
3. Export a runtime JS file that emits the Shaderfrog API steps that create the graph nodes
4. Precompile a bunch of files based on known scene parameters (unlikely to be welcomed by Three community, unrealistic for Babylon)

For options 1-3, there is the possible optimization step we could apply. If there is no three.js material, it's possible we might not need the shaderfrog API source code and compilation step, and can simply export the static GLSL.

We also have the option to let users choose: If there is no three.js material, you could either export the API code (gives end developers more control over modifying shaders), or you could export the mashed together GLSL and not require Shaderfrog source code.

It feels weird that some export paths might require Shaderfrog core libraries to be included in your engine (like if you have a three.js material) and some export paths might not require it in the optimization case. As an end consumer, it seems weird to ask someone to sometimes include Shaderfrog core libraries, and sometimes not depending on which shader you're exporting.

I don't know why you would take the time to read this whole post. If you did, what are your preferences for exporting formats? Would you use an export that requires you to include shaderfrog libraries?

[Bug-Reaper]

Banger read bro, tysm!

---

Background

Been a three.js enjoyer for almost a decade and enjoyed the original shader frog for many years! As I move to launch my own software that revolves on three.js I have some extra time to contribute. Excited to see shaderFrog v2 bring more material diversity to the three.js space and pillar an ecosystem to share those creations.

Have put a bunch of skillpoints into code but am pretty smooth brained on GLSL. I've read through this reply a couple of times and also had a discussion on the THREE.js discord try and fully understand. My apologies in advance if my shallow GLSL knowledge results in additional naive suggestions.

Takeaways

ShaderFrog sometimes relies on GLSL code ripped from locally generated native THREE.js materials. Because it's modifying generated code, you cannot easily bake the result shaders as they contain scene-specific and hardware-specific nuance. Such as number of lights in the scene, GL version, and likely many others.

Miscellaneous

Blows my mind that the native materials get re-compiled as lights are added/removed from the scene. Rummaged a bit and discovered:

1. The renderer controls and triggers recompilation of native THREE.js materials as lights get added/removed from the scene.
2. You can get THREE.ShaderMaterial()s to recompile on light changes with a boolean flag. Looks like there's some additional re-compilation magic you can make happen on these guys too.
3. The THREE.RawShaderMatierial() cannot easily tap into necessary re-compilation hooks.

Solution Preference => Option 2️⃣ pls

My vote is for 2 or 3.

2. Export a runtime JS file like you have, that is parsed and compiled at runtime, where most of the data is static/JSON
3. Export a runtime JS file that emits the Shaderfrog API steps that create the graph nodes

For example could create a library that exposes the needed dependencies. Then from the runtime JS file emit some warning/error that tells the user to npm install the library(s) if they're not present and optionally links to a docs page.

I think the runtime JS file is the most portable, developer friendly and easiest to integrate. I especially like that:

• It saves you from 1 level of callback hell since it eliminates the need to fetch and consume a json file.
• Trivial to add custom module deps like this in any context, especially given THREE.js is module only.

Solution Considerations

In no particular order some other loose ideas.

Notable Example

Discovered this project for a vanilla-threejs-hologram-material that bakes the material into a runtime JS file like this. It's similar to my original snippet with a couple of things I think I like. Notably it allows for default uniform overrides to be passed into the material constructor input param object. Makes the custom material really feel like a first class native THREE.js material.

How Runtime Compilation Magic

I'm super keen to look into how native THREE.js materials handle runtime-specific GLSL generation magic. Suspect there may be an opportunity to inject into the material setup code instead of into the fully generated shader code. Ideally can still leverage some of the GLSL into an AST stuff that sounds sexy as all hell. I guess the difference is it would do so inside blocks of GLSL used by the material setup code instead of on the fully compiled GLSL?

---

Well this got long fast lol. I have even more thoughts but most of the valuable stuff has been expressed. Hopefully next time I check in may have some code that tests some of the ideas here.