Doc : Access to instance positions in NodeMaterial

Makio64 commented 1 month ago

Description

I coudnt find the right way to access or use instance properties in TSL from the doc ( Three.js-Shading-Language ) or examples.

Would be nice to get the explanation on how to access to the ID and instanceMatrix / instanceColorMatrix and i would be happy to create simple examples and update the documentation.

Solution

Update the doc for TSL including accessors for instance / batch
Add minimal examples : The color of the material change depending of the x y z axis of the instance

Alternatives

Another minimal example : the scale of the instance change depending of the luminosity of the instanceColorMatrix

Additional context

No response

cmhhelgeson commented 1 month ago

Hi @Makio64,

To access the instance of a mesh within a TSL function, you need to use the instanceIndex node. In most current examples, instanceIndex is used within compute shaders to represent the ID of the current compute thread. However, the value of instanceIndex is context-dependent.

In a compute shader, instanceIndex represents a thread ID.
However, In a vertex or fragment shader, instanceIndex represents the instance ID that the current vertex belongs to.

Here's a quick example I wrote up demonstrating how to use instanceIndex within a TSL vertex shader.


const geometry = new THREE.BoxGeometry(0.1, 0.1, 0.1);
const texture = new THREE.TextureLoader().load( 'textures/crate.gif' );
texture.colorSpace = THREE.SRGBColorSpace;
const material = new THREE.MeshBasicNodeMaterial( {map: texture});

const uCircleRadius = uniform(1.0);
const uCircleSpeed = uniform( 0.5 );
const instanceCount = 80;
const numCircles = 4;
const meshesPerCircle = instanceCount / numCircles;

material.positionNode = Fn(() => {
        // Multiply elapsed time by speed
    const time = timerLocal().mul( uCircleSpeed );

        // Each cube is 1 of 20 within a concentric circle. Get index of cube within a circle
    const instanceWithinCircle = instanceIndex.remainder(meshesPerCircle );

        // Get index of the circle itself
    const circleIndex = instanceIndex.div( meshesPerCircle ).add(1);

       // Radius of each circle increases
    const circleRadius = uCircleRadius.mul(circleIndex);

    // Normalize instanceIndex to range [0, 2*PI]
    const angle = float(instanceWithinCircle).div(meshesPerCirlce).mul(PI2).add( time ); 
    const circleX = sin( angle ).mul( circleRadius );
    const circleY = cos( angle ).mul( circleRadius );

    const scalePosition = positionGeometry.mul( circleIndex );

    const rotatePosition = rotate(scalePosition, vec3( timerLocal() , timerLocal(), 0.0));

    const newPosition = rotatePosition.add( vec3( circleX, circleY, 0.0 ));
    return vec4( newPosition, 1.0 );
})(); 

mesh = new THREE.InstancedMesh( geometry, material, instanceCount );
scene.add( mesh );

renderer = new THREE.WebGPURenderer({ antialias: false })
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.setAnimationLoop( animate );
document.body.appendChild( renderer.domElement );
//

const controls = new OrbitControls( camera, renderer.domElement );
controls.minDistance = 1;
controls.maxDistance = 20;

const gui = new GUI();
gui.add( uCircleRadius, 'value', 0.1, 3.0, 0.1 ).name( 'Circle Radius' );
gui.add( uCircleSpeed, 'value', 0.1, 3.0, 0.1 ).name( 'Circle Speed' );

And the output:

https://github.com/user-attachments/assets/e5217fda-1989-415d-8e90-bce5511051d0

For modifying the color of an instance, you can simply apply 'instanceColor' as an attribute to your geometry, then access that attribute within your material's fragmentNode or colorNode. This is what I currently do, though I'm not sure if there's a built-in TSL solution for instanceColors.

const instanceColorArray = new Float32Array( instanceCount * 4 );
for ( let i = 0; i < instanceColorArray.length; i++ ) {

    instanceColorArray[i * 4 + 0] = Math.random();
    instanceColorArray[i * 4 + 1] = Math.random();
    instanceColorArray[i * 4 + 2] = Math.random();
    instanceColorArray[i * 4 + 3] = Math.random();

}

geometry.setAttribute( 'instanceColor', new THREE.InstancedBufferAttribute( instanceColorArray, 4 ) );
material.fragmentNode = attribute('instanceColor');

https://github.com/user-attachments/assets/07490001-bafb-4d6d-9eb9-d859ca48004d

With this method, you also have the added flexibility of modifying your instanceColor within a compute shader. All you need to do is change your InstancedBufferAttribute to a StorageInstancedBufferAttribute, thus making it a datatype that is accessible as a storage buffer within a compute shader.


let computeColor;

const instanceColorArray = new Float32Array( instanceCount * 4 );
for ( let i = 0; i < instanceColorArray.length; i++ ) {

    instanceColorArray[i * 4 + 0] = Math.random();
    instanceColorArray[i * 4 + 1] = Math.random();
    instanceColorArray[i * 4 + 2] = Math.random();
    instanceColorArray[i * 4 + 3] = Math.random();

}

// Make attribute accessible as a storage buffer within a compute shader.
const instanceColorAttribute = new THREE.StorageInstancedBufferAttribute( instanceColorArray, 4 );
geometry.setAttribute( 'instanceColor', instanceColorAttribute );

material.fragmentNode = attribute('instanceColor');

computeColor = Fn(() => {

    const instanceColor = storage( instanceColorAttribute, 'vec4', instanceCount );

    const r = sin( timerLocal().add(instanceIndex) );
    const g = cos( timerLocal().add(instanceIndex));
    const b = sin( timerLocal() );

    instanceColor.element( instanceIndex ).assign(vec4(r, g, b, 1.0));

})().compute( instanceCount );

function render() {
     renderer.render( scene, camera)
     renderer.compute( computeColor );
}

https://github.com/user-attachments/assets/24548fdc-f6df-4380-aef8-f24941bc42a0

Makio64 commented 1 month ago

Thanks @cmhhelgeson, nice examples, very instructive !

In my case I still need to access to the : instanceMatrix to know the position of the instance in the worldSpace ( basically to apply position modifier on the vertex depending of the position of the vertex in worldspace )

I tried to access it using {instance} from 'three/tsl' but there is a logic I cant figure out.

The easy solution would be to do a InstancedBufferAttribute or StorageInstancedBufferAttribute with the data I need ( position of the instance ) but as we already have an instanceMatrix system so I would like to understand how to access and use it.

cmhhelgeson commented 1 month ago

Thanks @cmhhelgeson, nice examples, very instructive !

In my case I still need to access to the : instanceMatrix to know the position of the instance in the worldSpace ( basically to apply position modifier on the vertex depending of the position of the vertex in worldspace )

I tried to access it using {instance} from 'three/tsl' but there is a logic I cant figure out.

The easy solution would be to do a InstancedBufferAttribute or StorageInstancedBufferAttribute with the data I need ( position of the instance ) but as we already have an instanceMatrix system so I would like to understand how to access and use it.

I myself wasn't even aware there was an instance node 😅.

I'm not quite sure what the requirements of your application are, but I think a good place to start would be using the positionWorld node (I can get back on whether the modelWorldMatrix used in positionWorld updates to account for instancing). As you mentioned, you could also just replace the current InstanceMatrix InstancedBufferAttribute with a StorageInstancedBufferAttribute, thus making it accessible as a storage node within TSL shaders. Look at webgpu_compute_geometry.html to see how StorageBufferAttributes can easily replace existing buffer attributes.

Makio64 commented 1 month ago

@z4122 @sunag do you mind sharing an example of how to use the InstanceNode / access to the instance matrix from the tsl ? 🥺

Thanks !

sunag commented 1 month ago

Currently InstanceNode does not have feature to just export instanceMatrixNode property, maybe you will have to open the source code and use the separate code part for now.

import { buffer, instanceIndex } from 'three/tsl';

// if ( instanceMesh.count <= 1000 )
const node = buffer( instanceMesh.instanceMatrix.array, 'mat4', instanceMesh.count ).element( instanceIndex );

https://github.com/mrdoob/three.js/blob/410f73742f0043540857a60d3827e4635875ec90/src/nodes/accessors/InstanceNode.js#L41-L73

mrdoob / three.js