GPU Particles

[HifiExperiments]

Closes #832

Our current ParticleEffect entities have various limitations: 100,000 particles / entity, only billboarded quads, and only euler integration (position + velocity + acceleration).

This PR adds a new entity type, ProceduralParticleEffect, which gives you fine-grained control over a GPU particle system using procedural shaders for updating (per-particle updates via a fragment shader, which we can eventually convert to a compute shader) and rendering (via vertex and fragment shaders), plus higher per-system limits.

The old ParticleEffect entity is left as is, but almost all of the same functionality is possible with a ProceduralParticleEffect (everything except mesh shape emitters since we don't currently have all the mesh triangles on the GPU). Later, we could consider deprecating it and providing a drop in ProceduralParticleEffect replacement (which could also support skinned/animated mesh emitters). When you go to make a new particle entity in Create, it prompts you for which type you want.

On the rendering side, the particles are controlled by particleRenderData which acts the same way as our current procedural materials on entities. The vertex shader must implement vec3 getProceduralVertex(const int particleID), which returns the world space position of the vertex. By default, 1 triangle is drawn per particle, so that function will be run 3 times per particle (with different gl_VertexIDs), but this can be controlled by numTrianglesPerParticle (see the 3rd example below). The fragment shader must implement one of the same functions as our existing procedural materials depending on version: vec3 getProceduralColor(), float getProceduralColors(inout vec3 diffuse, inout vec3 specular, inout float shininess), float getProceduralFragment(inout ProceduralFragment proceduralData), or float getProceduralFragmentWithPosition(inout ProceduralFragmentWithPosition proceduralData). In the fragment shader, particleID is available as a global variable. One difference between procedural particles and procedural materials on other entities is that they do not have per-fragment normals or uvs defined by default, you must calculate them yourself if you want lit or textured particles (see the 3rd example below).

Per-particle updates are optional. They can be used to store persistent per-particle data across frames or reduce calculations in the rendering shaders. To activate updates, numUpdateProps must be > 0 (max is 5) and you must provide a fragment shader as part of particleUpdateData. Each "update prop" is a per-particle vec4 of float values. The update shader must define void updateParticleProps(const int particleID, inout ParticleUpdateProps particleProps). particleProps will contain as many update props as you specified (e.g. if numUpdateProps == 3, you'll have particleProps.prop0, particleProps.prop1, and particleProps.prop2). They will initially contain all 0s, and then will persist any values from previous frames. You can look up these properties during rendering with vec4 getParticleProperty(const int propIndex, const int particleID) in both the vertex and fragment shaders.

Like the old ParticleEffect, transparent particles (particleTransparent == true) render with additive blending since they are unsorted.

Like procedural vertex shaders on material entities, the dimensions of the entity are used for culling.

Notes + follow up work:

• Shaders are hard. We should create a library of simple effects that can be driven by scripts/uniforms and add them to Create as presets.
• Currently, you have to do some heavy lifting in your vertex shader to create shapes, especially if you want lighting (which require you to calculate normals). We could probably add a property like "particleShape" with a "custom" mode that would mirror this behavior, but also provide things like "sphere", "billboarded quad/triangle", and "model" (although that'll be trickier) to easily draw each particle as those shapes (which could then be adjusted in the vertex shader).

Examples

Cube of oscillating, unlit, billboarded triangles, no update:

Entities.addEntity({
    type: "ProceduralParticleEffect",
    position: Vec3.sum(MyAvatar.position, Vec3.multiply(4, Quat.getFront(MyAvatar.orientation))),
    dimensions: 3,
    numParticles: 10000,
    numTrianglesPerParticle: 1,
    numUpdateProps: 0,
    particleRenderData: JSON.stringify({
        version: 1.0,
        vertexShaderURL: "https://gist.githubusercontent.com/HifiExperiments/1c1f83814b332782b590f44eca329fc4/raw/51753492e8634858239d57d6b8fb0f136a9f49af/proceduralParticle.vs",
        fragmentShaderURL: "https://gist.githubusercontent.com/HifiExperiments/7def54504362c7bc79b5c85cd515b98b/raw/93b3828c2ec66b12b789a625dd141f533c595ede/proceduralParticle.fs",
        uniforms: {
            radius: 0.03
        }
    })
})

Same cube of oscillating, unlit, billboarded triangles, but with the oscillation in the update (computed once per particle instead of once per vertex):

Entities.addEntity({
    type: "ProceduralParticleEffect",
    position: Vec3.sum(MyAvatar.position, Vec3.multiply(4, Quat.getFront(MyAvatar.orientation))),
    dimensions: 3,
    numParticles: 10000,
    numTrianglesPerParticle: 1,
    numUpdateProps: 1,
    particleUpdateData: JSON.stringify({
        version: 1.0,
        fragmentShaderURL: "https://gist.githubusercontent.com/HifiExperiments/9049fb4a8dcd2c1401ff4321103dce16/raw/4f9474ed82c66c1f94c1055d2724af808cd7aace/proceduralParticleUpdate.fs",
    }),
    particleRenderData: JSON.stringify({
        version: 1.0,
        vertexShaderURL: "https://gist.github.com/HifiExperiments/5dda24e28e7de1719e3a594d81306343/raw/92e0c5b82a9fa87685064cdbab92ed0c16f49f94/proceduralParticle2.vs",
        fragmentShaderURL: "https://gist.github.com/HifiExperiments/7def54504362c7bc79b5c85cd515b98b/raw/93b3828c2ec66b12b789a625dd141f533c595ede/proceduralParticle.fs",
        uniforms: {
            radius: 0.03
        }
    })
})

Swarm of 3D particles simulating gravity, colored with their normals, lit:

Entities.addEntity({
    type: "ProceduralParticleEffect",
    position: Vec3.sum(MyAvatar.position, Vec3.multiply(6, Quat.getFront(MyAvatar.orientation))),
    dimensions: 3,
    numParticles: 10000,
    numTrianglesPerParticle: 6,
    numUpdateProps: 3,
    particleUpdateData: JSON.stringify({
        version: 1.0,
        fragmentShaderURL: "https://gist.github.com/HifiExperiments/5e87462f7be127882a1657c463c3638f/raw/d7cd4b3e83b0bfe740e5c93b65f2455b38b8341f/proceduralParticleUpdate2.fs",
        uniforms: {
            lifespan: 3.0,
            speed: 2.0,
            speedSpread: 0.25,
            mass: 50000000000
        }
    }),
    particleRenderData: JSON.stringify({
        version: 3.0,
        vertexShaderURL: "https://gist.github.com/HifiExperiments/f0a86ecd37dedde3a2d187373a9117d1/raw/1fe678f0e28130839deab4fff8d04c279cdd1581/proceduralParticle3.vs",
        fragmentShaderURL: "https://gist.github.com/HifiExperiments/2174e4cbff17fbeb4390353e6791920f/raw/fd6abf3f8bb21fc3fbd58e2aa0c47fd895352789/proceduralParticle2.fs",
        uniforms: {
            radius: 0.03,
            lifespan: 3.0
        }
    })
})

Funding

This project is funded through NGI0 Entrust, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.

[vegaslon]

certainly looks interesting https://gyazo.com/f33d97de2202fc94c399e919ca1ac3e6 find that the particles completely disapear if not looking close to emitter, which I am sure is wise for performance reasons but kind of a bummer.

[HifiExperiments]

@vegaslon ah, forgot to mention, they use their dimensions directly as their bounding box, like our existing procedural vertex shaders via material entities, since we don't know on the CPU where the shader is going to place the particles. so you can just make their dimensions bigger and they won't get culled like that.

once we merge this, I will put up a page in the docs with these details

[SilverfishVR]

a minor alignment issue:

[JulianGro]

You can probably just get rid of the "Num" for the numbers. If someone is unsure, they can look at the tooltip.

[HifiExperiments]

here's two more basic examples (these are scripts that set up the entities), these also show how we'd re-implement some of the existing CPU particle properties (emitDimensions, speed, speedSpread, acceleration):

Particles bouncing off a 3D plane, driven by a plane entity

Particles attracted to another entity

[vegaslon]

https://i.gyazo.com/d93a81979e5dbe7ce76b3249a3445889.mp4 ok that is seriously really neat

[HifiExperiments]

I've fixed the typo and added a default effect to create. unless anyone has any more thoughts this is ready to be merged from my side!

[vegaslon]

Always hard to test something brand new like this and so I have no complaints with what I have found. I figured this could only be held up by things like what standard of ease of usability does it need to go for or some such. Before it is considered merge able.

My opinion is this is good enough right now to merge and any addition assistance to make it more usable can be created with addition examples down the line and a app to help, like what happened to material entities, before the create app is modified.