THREE.Instances

[mrdoob]

I've been toying with this API recently.

var mesh = new THREE.Mesh( geometry, material );

var instances = new THREE.Instances( mesh );
instances.objects.push(
    new THREE.Object3D().setPosition( 1, 1, 1 ).setRotation( 0, 1, 0 ),
    new THREE.Object3D().setPosition( 2, 2, 2 ),
    new THREE.Object3D().setPosition( 3, 3, 3 ).setScale( 2, 1, 1 )
);
scene.add( instances );

Based on ANGLE_instanced_arrays. (~50%)

Thoughts?

[zz85]

:+1: for instanced arrays.

However, I'm not sure about the instances.objects part though. Since instanced arrays are use to create eg 100k objects, don't think its a good idea to create 100k Object3D()s (the same reason behind BufferGeometry.

Rather,

var mesh = new THREE.Mesh( geometry, material );

var instances = new THREE.Instances( mesh );
instances.position = new Float32Array();
instances.rotation = new Float32Array();
instances.scale = new Float32Array();

scene.add( instances );

[zz85]

Also, I'm not sure how well is THREE.Instances( mesh ); going to play round with other types, eg.

new THREE.Instances( pointCloud );
new THREE.Instances( line );

Or whether it'd be better to do

var mesh = new THREE.InstancedMesh( geometry, material );
var pointCloud = new THREE.InstancedPointCloud( geometry, material );
var line = new THREE.InstancedLine( geometry, material );

scene.add( mesh );
scene.add( pointCloud );
scene.add( line );

...

[mrdoob]

However, I'm not sure about the instances.objects part though. Since instanced arrays are use to create eg 100k objects, don't think its a good idea to create 100k Object3D()s (the same reason behind BufferGeometry.

What about...

var instances = new THREE.Instances( mesh );
instances.matrices.push(
    new THREE.Matrix4().makeTranslation( 1, 1, 1 ),
    new THREE.Matrix4().makeTranslation( 2, 2, 2 ),
    new THREE.Matrix4().makeTranslation( 3, 3, 3 )
);
scene.add( instances );

[zz85]

hmm... i'm not sure if this make sense to me right now...

in @toji instanced monkey example, there are 3 elements - the positions for mesh instances, the individual positions, and individual colors.

so mesh position offsets maps to a geometry, and both positions and colors could be individual FloatArrays.

perhaps

monkey = new Geometry(); // usual geometry
material = new Material(); // could be the usual material or custom shader material
instances = new THREE.InstancedMesh( monkey, material, 10000 );

instances.addAttribute('color', 3);
instances.addAttribute('position', 3);

scene.add( instances )

?

[TheJim01]

What about...

var instances = new THREE.Instances( mesh );
instances.matrices.push(
    new THREE.Matrix4().makeTranslation( 1, 1, 1 ),
    new THREE.Matrix4().makeTranslation( 2, 2, 2 ),
    new THREE.Matrix4().makeTranslation( 3, 3, 3 )
);
scene.add( instances );

This would require instances.matricies and instances.materials to be the same length, which--if left to the user--might not always be the case. It might be more convenient to:

instances.addInstance( { matrix: new THREE.Matrix4().makeTranslation(1,1,1), material: new THREE.MeshBasicMaterial( { color: 'red' } ) } );

The API handles metadata allocation, and becomes a little more dynamic, allowing it to accept other fields.

[mrdoob]

Hmmm... I was thinking that THREE.Instances would have just one material for all the instances though.

[TheJim01]

I get what you're saying, and it makes sense. It avoids a bunch of headaches surrounding material sharing/instancing inside the instance array. If you have an instance array of green cubes, and want some of them to be red, it's easier to make another instance array of red cubes, and nothing is stopping me from re-using the geometry over again.

    var cube = new THREE.BoxGeometry( 5, 5 );
    var mesh1 = new THREE.Mesh( cube, new THREE.MeshBasicMaterial( { color: 'green' } ) );
    var instances1 = new THREE.Instances( mesh1 );
    instances1.matrices.push(
        new THREE.Matrix4().makeTranslation( 1, 1, 1 ),
        new THREE.Matrix4().makeTranslation( 2, 2, 2 ),
        new THREE.Matrix4().makeTranslation( 3, 3, 3 )
    );
    scene.add( instances1 );
    var mesh2 = new THREE.Mesh( cube, new THREE.MeshBasicMaterial( { color: 'red' } ) );
    var instances2 = new THREE.Instances( mesh2 );
    instances2.matrices.push(
        new THREE.Matrix4().makeTranslation( 4, 4, 4 ),
        new THREE.Matrix4().makeTranslation( 5, 5, 5 ),
        new THREE.Matrix4().makeTranslation( 6, 6, 6 )
    );
    scene.add( instances2 );

:+1:

[Baccanno]

Looking at the proposal I'm very more keen on something keeping same logic and syntax than the other mesh stuff.

Here I need to calculate manually my transform matrix but everything is ready in the Object3D class for manipulation object transformation ( and everyone knows how to ), Would it be possible to perhaps do something like

    var cube = new THREE.BoxGeometry( 5, 5 );
    var mesh = new THREE.Mesh( cube , new THREE.MeshBasicMaterial( { color: 'green' } );
    var instance1 = new THREE.InstancedMesh( mesh );
    var instance2 = new THREE.InstancedMesh( mesh );

   myParentOnACliff.add( instance1 );
   instance1.rotation.x = Math.PI;

  myParentUnderTheSea.add( instance2 );
  instance2.position.setX( 50 );

   var mesh2 = new THREE.Mesh( cube , new THREE.MeshBasicMaterial( { color: 'red' } );
   for (var i = 0; i < 1000; ++i){
      myOtherParent.add( new THREE.InstancedMesh( mesh2 );
   }

Then when the instance is added under the hood we calculate the correct transform matrix.

This would add much more control over instances, so they can be moved, removed, switched from one parent to an other, well any scene graph use case without having to redo everything that's already done.

If I'm not wrong I understand that the proposal wants to keep objects in memory low, but It would be great to find a compromise between memory performances and usability.

[mrdoob]

You could do something like this:

var object1 = new THREE.Object3D();
object1.position.x = 10;

var object2 = new THREE.Object3D();
object2.rotation.x = 1.57;

var object3 = new THREE.Object3D();
object3.scale.x = 2;

var instances = new THREE.Instances( mesh );
instances.matrices.push(
    object1.matrix,
    object2.matrix,
    object3.matrix
);
scene.add( instances );

[Baccanno]

And updating the object3D matrices would work ?

[mrdoob]

Not initially... I'm looking at Object.observe() though.

[Baccanno]

Is this solution well suited for independently moving objects then ?

[mrdoob]

It kind of allows what you want to do and, at the same time, it can be memory efficient.

[Baccanno]

In order to get the parent transform of objects shouldn't we use .matrixWorld after calling updateMatrixWorld() ?

Then couldn't we have a instances.updateMatrices() ?

So the objects would use normal API and scene graph placement, and at each frame we would update the instanced matrices

[mrdoob]

What I am trying to avoid is having to create a Object3D per instance.

[Baccanno]

I really here the memory concern, but in some case someone may do it because it's much simpler to manage the object states and position accordingly to their "real position" in the scene graph.

Was just trying to figure out if both approaches were compatible. May be doing a simple wrapper on a demo will do the trick on top of this API.

[mrdoob]

Was just trying to figure out if both approaches were compatible. May be doing a simple wrapper on a demo will do the trick on top of this API.

Yeah, that's the idea. Figure out a base API that one can build on top.

[benaadams]

I'm have something related, though its raw buffergeometry based as that's how I roll :)

I'm finding the hardest bit being coming up with good names, and not cause an explosion of types, or word combos.

Question on dynamic vs non-dynamic (static vs updateable):

new InstancedBufferAttribute() new InstancedDynamicBufferAttribute()

or

new InstancedBufferAttribute({ meshPerAttribute: 1, dynamic: false }) // defaults new InstancedBufferAttribute({ meshPerAttribute: 1, dynamic: true })

I'm tending towards second as including a instancedbuffer means you are now using a different draw call type.

[Baccanno]

I agree with you with the second naming, as it fits more the current way of doing with geometry.

[benaadams]

Bit random, but is only 1 triangle: http://prealpha.ageofascent.com/examples/webgl_buffergeometry_instancing.html

Still working on stuff...

[benaadams]

Ok that looks like it works; will try something more interesting/normal for the indexed version, like using models :)

[benaadams]

Less interesting visually but indexed and textured cube (1) instanced 5000 times and dynamically updating the local rotations (CPU->GPU): http://prealpha.ageofascent.com/examples/webgl_buffergeometry_instancing_dynamic.html

Seem ok?

[benaadams]

Just highlighting proposed BufferGeometry approach, set up:

var instances = 5000;
var geometry = new THREE.InstancedBufferGeometry();
// per mesh data
var vertices = new THREE.BufferAttribute( new Float32Array( [ ... ] ) );
geometry.addAttribute( 'position', vertices );
var uvs = new THREE.BufferAttribute( new Float32Array( [ ... ] ) );
geometry.addAttribute( 'uv', uvs );
var indices = new Uint16Array( [ ... ] ) );
geometry.addAttribute( 'index', new THREE.BufferAttribute( indices, 1 ) );
// per instance data
var offsets = new THREE.InstancedBufferAttribute( new Float32Array( instances * 3 ), 3, 1, false ); // not dynamic
... // set up data
geometry.addAttribute( 'offset', offsets ); // per mesh translation
rotations = new THREE.InstancedBufferAttribute( new Float32Array( instances * 4 ), 4, 1, true ); // is dynamic
... // set up data
geometry.addAttribute( 'rotation', rotations ); // per mesh rotation
var mesh = new THREE.Mesh( geometry, material );
scene.add( mesh );

Updating rotations CPU side and transfering

function render() {
    .... // other stuff
    for ( var i = 0, ul = rotations.length / 4; i < ul; i++ ) {
        .... // some calcs
        rotations.setXYZW( i, currentQ.x, currentQ.y, currentQ.z, currentQ.w );
    }
    rotations.needsUpdate = true;
}

Comments welcome, have PR ready for both examples, just running through some further tests

[benaadams]

Still needs some checks so it doesn't break stuff...

[Baccanno]

Did you testcase with MeshFaceMaterial ?

[benaadams]

Yes in terms of not breaking: http://prealpha.ageofascent.com/examples/webgl_materials.html and http://prealpha.ageofascent.com/examples/webgl_materials2.html (I'm guessing its one on of the pages)

With the current THREE shaders instancing doesn't give you anything as such; for example you could use color as an instanced value and draw 10 instances of a mesh and they would all draw and have different colors but they would all draw in same place. This is why in the two examples I use RawShaderMaterial.

To change that shaders probably needs some debate on best approach; obviously you want to #ifdef on whether they are instanced; but should it be maxtrices that the instance adjustments are communicated as or smaller parts; like position. rotation, scale. For fixed objects a second object matrix would be ok and introduce a second matrix multiply in the shader with a 16 float extra data per object; but for updating objects (changing position or rotation) then that becomes a bit heavy weight; and position, rotation, and an all axis uniform scale can be communicated in 8 floats and then you can update rotation and position independently.

So not sure what the preferred approach would be. Api-wise it could be all hidden away.

@mrdoob any preferences?

Not doing matrixes would mean loosing some of the more unusual possibilities like skew, not uniform scaling etc and need an addition (position), multiplication (scale) and quaternion multiply (rotation) x2 for normals in the shader and use 2 extra 4 float attributes, and a simple load to GPU.

Doing a matrix would mean doing all the conversion to matrix on the CPU for position, scale and rotation quaternion for each instance and need a matrix multiply x2 for normals in the shader and use 4 extra 4 float attributes, and a load to GPU post CPU conversion.

[mrdoob]

Looking good @benaadams! Thanks for hacking this up!

I'll have a closer look over the weekend.

Did you testcase with MeshFaceMaterial ?

I wouldn't worry about MeshFaceMaterial. I'm trying to deprecate it in the smoothest way...

[benaadams]

PR is #6242 also added interleaved buffers; and normal, dynamic, instanced, interleaved and interleaved instanced can be all used together for different buffer types within the same geometry. I think the usage is clean; but WebGLRenderer has a little too many if blocks for my liking; can't see a easy refactor; maybe move the warning tests to a separate function.

[dubejf]

@mrdoob Is this design still relevant after @benaadams work on InstancedBufferGeometry?

[mrdoob]

I think so yeah... InstancedBufferGeometry is a little bit too advanced. Would be nice to find something that sits on top.

[MyCADDev]

This is very much needed functionality. Could we also have instances of instances?

[emnh]

@benaadams Could you make a test case where you use also normal Geometry or BufferGeometry together in the scene with InstancedBufferGeometry?

I am getting the following error trying to use InstancedBuffer together with other geometries: Error: WebGL: drawElements: at least one vertex attribute divisor should be 0

I need to explicitly set 0 divisor for each non-instanced attribute, then it works fine: extension.vertexAttribDivisorANGLE(programAttribute, 0);

This is with Firefox on Windows.

[benaadams]

@emnh should work fine? Will alter examples.

Normal meshes shouldn't be effected and for Instanced stuff you need to create an new THREE.InstancedBufferGeometry and add the buffers to that.

Mostly they will be normal BufferAttribute and then one or more new THREE.InstancedBufferAttribute( new Float32Array( instances * 3 ), 3, 1, false )

[emnh]

@benaadams Hmm.. I'm having no luck so far reducing my problem to a simple test case. Seems to work fine having both InstancedBufferGeometries and other geometries in any simple test case I've come up with.

I'm seeing the problem in my game, which is a bit complex by now, so it's hard to say exactly what must be present for the problem to occur. The symptom is that as soon as an InstancedBufferGeometry is rendered, the ground, which is a PlaneBufferGeometry, disappears and I get "Error: WebGL: drawElements: at least one vertex attribute divisor should be 0".

You can see the problem in my game here. You might have to move the camera a bit using the arrow keys for the InstancedBufferGeometries to render.

[benaadams]

The symptom is that as soon as an InstancedBufferGeometry is rendered, the ground, which is a PlaneBufferGeometry, disappears and I get "Error: WebGL: drawElements: at least one vertex attribute divisor should be 0".

Hmm, this may be due to an attribute remaining flagged as active when it renders. @mrdoob has been a lot of churn in the attribute activation/deactivation code, any pointers where to check?

[mrdoob]

To be honest, I yet don't understand how InstancedBufferGeometry works. So I would need a bit of help there...

[benaadams]

Was going to answer but realised I probably don't know what's happening currently :smile:

Will have a dig through the code...

[hector-del-rio]

@benaadams is there any progress on this? @mrdoob is the current API final?

[Mevedia]

It's a great and very important performance boost. Maybe an interface to access instances temporary would make it easier to work with, something like:

motherMesh.instance( index ).rotateX( ... )

While the instance method would return a "THREE.MeshInstance" singleton that get's this index assigned, and properties are read from buffer on access. Then maybe a method on it to spawn a "THREE.MeshInstance" object, to work with it like a basic mesh, but being able to delete or reassign the "THREE.MeshInstance" to another index.

[mrdoob]

Kind of similar to what I suggested on the other thread... https://github.com/mrdoob/three.js/pull/10093#issuecomment-260165930 👍

[mrdoob]

Closing in favor of #17505.