Animating uniforms in ShaderMaterial

[dbismut]

Hey, I'm quite new to all this, but I thought this might be a good occasion to get more familiar with THREE. I'm trying to replicate this demo here using react-three-fiber. https://tympanus.net/Development/DistortionHoverEffect/

From the demo source the material is built and animated as below with GSAP (dispFactor value animates from 0 to 1):

var mat = new THREE.ShaderMaterial({
    uniforms: {
        effectFactor: { type: "f", value: intensity },
        dispFactor: { type: "f", value: 0.0 },
        texture: { type: "t", value: texture1 },
        texture2: { type: "t", value: texture2 },
        disp: { type: "t", value: disp }
    },

    vertexShader: vertex,
    fragmentShader: fragment,
    transparent: true,
    opacity: 1.0
});

// animation with GSAP
parent.addEventListener(evtIn, function(e) {
    TweenMax.to(mat.uniforms.dispFactor, speedIn, {
        value: 1,
        ease: easing
    });
});

I've tried to do the same with react-three-fiber and react-spring but nothing happens or nothing shows. Any idea what I'm doing wrong?

const [hovered, setHover] = useState(false)
const hover = useCallback(() => setHover(true), [])
const unhover = useCallback(() => setHover(false), [])
const { progress } = useSpring({ progress: hovered ? 1 : 0 })

return (
    <anim.mesh {...props} onHover={hover} onUnhover={unhover}>
      <planeBufferGeometry name="geometry" args={[3.8, 3.8]} />
      <anim.shaderMaterial
        name="material"
        uniforms={{
          effectFactor: { type: 'f', value: intensity },
          dispFactor: { type: 'f', value: progress },
          texture: { type: 't', value: texture1 },
          texture2: { type: 't', value: texture2 },
          disp: { type: 't', value: displacement }
        }}
        {...HoverShader}
        transparent
        opacity={1}
      />
    </anim.mesh>
);

[drcmda]

I don't see anything wrong there, i guess it's something else. Could you set up a codesandbox?

But either way, you can either pierce into the uniform (given that you have created it previously):

<animated.material uniforms-dispFactor-value={animatedValue} />

or sometimes the shader has variables, here's a demo where i bind the scrolloffset to a glitch shader:

https://codesandbox.io/embed/y3j31r13zz

[dbismut]

Sure, at the moment it's very much inspired by your own sandbox. https://codesandbox.io/s/w5902o607

I'm pretty sure there's something wrong with value: progress since replacing it with a proper value between 0...1 shows the images.

[drcmda]

yes, you're putting a animatedValue into the uniforms. progress is not a value, it's a class. but either way, somegthing's wrong, will have to look into it.

[drcmda]

now it works: https://codesandbox.io/s/x37p4kk6p4

but that's still weird. it should be able to pierce into the value, but for some reason it only works when i write the object around it, for which it needs to interpolate.

i'll fix this tomorrow, then you will be able to just do: uniforms-xxxxx-value={animatedProp}

[dbismut]

Thanks! I think I understand a lot more how this works now, thanks again.

[drcmda]

i fixed the problem, here's the updated sandbox: https://codesandbox.io/s/x37p4kk6p4 it's released under @1.3.7

[drcmda]

PS. that the sprites are upside down has to do with a bug in current three, i've opened an issue and they've already fixed it, will go in their next patch. You can avoid it by resizing the images, must be divisible by 2 or something like that.

[dbismut]

Thanks! I don't know if you've noticed since they're rather abstract the images are upside down for some reason, from the source code I'm looking at, you might want to try:

 const { gl } = useThree()

  const [texture1, texture2, dispTexture] = useMemo(
    () => {
      const loader = new THREE.TextureLoader()
      const texture1 = loader.load(url1)
      const texture2 = loader.load(url2)
      const dispTexture = loader.load(disp)

      dispTexture.wrapS = dispTexture.wrapT = THREE.RepeatWrapping
      texture1.magFilter = texture2.magFilter = THREE.LinearFilter
      texture1.minFilter = texture2.minFilter = THREE.LinearFilter

      texture1.anisotropy = gl.capabilities.getMaxAnisotropy()
      texture2.anisotropy = gl.capabilities.getMaxAnisotropy()
      return [texture1, texture2, dispTexture]
    },
    [url1, url2, disp]
  )

(Anisotropy isn't needed but it's in the code, I'm not sure what it does even after reading the docs).

EDIT: sorry hadn't seen your PS, but magFilter apparently solves it.

[drcmda]

The issue was this one: https://github.com/mrdoob/three.js/issues/15896

[mysterybear]

Hey, sorry to bump this but I'm trying something very similar.

i'll fix this tomorrow, then you will be able to just do: uniforms-xxxxx-value={animatedProp}

This is what I'm after! But in the linked sandbox https://codesandbox.io/s/x37p4kk6p4 this isn't the approach that's actually used. In the sandbox you extend THREE.ShaderMaterial, write getters and setters, and then in App there's useFrame and ref mutations.

This is my code for piercing with animated values: https://codesandbox.io/s/github/mysterybear/r3f-image-crop/tree/spring-animated-shader-material?file=/src/App.js

It doesn't import animated and use it because every time I try to do that I run into some problem (edit: with no meaningful error; best case scenario is blank canvas, similar to OP). I've tried importing from @react-spring/three as well as /core, /web. Piercing with a regular array works, anything I try with animated I can't get working.

Cheers!

[mysterybear]

Ah my woes here are because of https://github.com/pmndrs/react-spring/issues/1430#issuecomment-823056058 so can disregard my previous comment... Couple things I needed to do:

1) wrap the result of drei's shaderMaterial in animated

2) pierce each element of a vector, e.g. uniforms-u_foo-value-x={to(...)} rather than uniforms-u_foo-value={to(...)}

Hopefully 2 will be fixed in react-spring soon tho!

[marceloavf]

@mysterybear How can I extend shaderMaterial with animated?

...
const WaveShaderMaterial = shaderMaterial(
...
)
extend({ WaveShaderMaterial })

const NoiseSphere = ({ theme }) => {
...
return (
    <animated.mesh ref={refMesh}>
      <animated.planeGeometry attach="geometry" args={[2, 2]} />
      <waveShaderMaterial ref={ref} attach="material" />
    </animated.mesh>
  )
}
...

Tried many ways but all returns errors to me, so I can't modify uniforms with spring, only lerp or direct input works

[mysterybear]

@marceloavf this is what worked for me in some code from many months ago, maybe you can try writing your code in a similar way and see if it works? Here it is:

import { animated } from "@react-spring/three"
import { shaderMaterial } from "@react-three/drei"
import { extend } from "@react-three/fiber"
import glsl from "glslify"
import * as THREE from "three"
import fragmentShader from "./fragment.glsl"
import vertexShader from "./vertex.glsl"

const CanvasImageMaterial = shaderMaterial(
  {
    u_mode: 0,
    u_texture: new THREE.Texture(),
    u_border_color: new THREE.Color(),
    u_border_thickness: new THREE.Vector2(0, 0),
    u_inset: new THREE.Vector4(0, 0, 0, 0),
    u_handle_length: 0.5,
    u_scale: 1,
  },
  glsl(vertexShader),
  glsl(fragmentShader)
)

export type CanvasImageMaterialImpl = {
  u_mode?: { value: number }
  u_texture?: { value: THREE.Texture }
  u_border_color?: { value: THREE.Color }
  u_border_thickness?: { value: THREE.Vector2 }
  u_inset?: { value: THREE.Vector4 }
  u_handle_length?: { value: number }
  u_scale?: { value: number }
} & JSX.IntrinsicElements["shaderMaterial"]

extend({ CanvasImageMaterial })

declare global {
  namespace JSX {
    interface IntrinsicElements {
      canvasImageMaterial: CanvasImageMaterialImpl
    }
  }
}

export const AnimatedCanvasImageMaterial = animated(
  (props: CanvasImageMaterialImpl) => <canvasImageMaterial {...props} />
)

There's also a blog post I wrote around the same time (hopefully not too out of date) on this topic https://bearjam.dev/blog/react-three-fiber-shaders-setup