Improved particle systems [$25]

[flo]

Terasology has a basic particle system implementation. It could be extended to support advanced features like particles that leave a trails.

Suggestions for additional effects

Ribbons

Examples of ribbon like effects in other games/engines:

• http://docs.runicgames.com/wiki/Missile_Particles_and_Ribbons
• https://docs.unrealengine.com/latest/INT/Engine/Rendering/ParticleSystems/Reference/TypeData/Ribbon/index.html

Those could be used to make spell like effects.

Force fields

Other advanced features could be force fields that change the direction of particles. Like for example force fields in blender:

• https://www.blender.org/manual/de/physics/force_fields.html

  Animated particles

Particles could have an animation that is based on a tile map and that plays in a loop.

Hints for new developers:

Familiarize yourself wit the Entity-System architecture of terasology: e.g. by reading ( https://github.com/MovingBlocks/Terasology/wiki/Entity-System-Architecture https://github.com/MovingBlocks/Terasology/wiki/Events-and-systems ) and by trying it out in practice.

The current particle system implementation can be found in the class BlockParticleEmitterSystem. It makes entities with a "BlockParticleEffect" compoent to a particle system.

A "prefab" that uses the component "BlockParticleEffect" is dustEffect.prefab. It can be spawned with the command "spawnPrefab dustEffect". Prefabs (at least those in modules) reload automatically, so you don't have to restart terasology. It is enough to spawn antother instance of the prefab to see the effect of your change in the json file.

[Cervator]

Quick reference to a force field that was done in the past: http://forum.terasology.org/threads/light-shadow.766/page-3#post-11413

Likewise there are a few similar usages here and there of particles, but centralizing the system + making better doc/tutorial on how to use it would be great :-)

[flo]

@Cervator that is a different kind of force field. With "force field" I didn't mean something visual, but just something that makes particles take differnt paths.

For example you could have a force field that pulls particle towards a center. Or makes particles start spinning around a center. Or the force field could push particles into a certain direction after they have flown into it.

[Cervator]

@flo oooohh! Okay. I get it now :-)

[Josharias]

I wonder if any of this is salvageable from Linus: https://github.com/LinusVanElswijk/Terasology/tree/feature/GeneralParticleSystems

[Remi05]

Hey, has any work been done on this issue to this day? I have a few ideas on how to improve the particle systems by using particle attractors for simple particle behaviours and I believe I could also implement a way to describe "force fields" that would allow for more complex behaviours.

[msteiger]

Hey Remi - thanks for your interest. We're currently in the GSOC 2016 proposal phase and some students have proposed to work on this subject over the summer. It has not yet been decided whether this will happen or not, though. There's another attempt for improved particle systems in WeatherManager: https://github.com/Terasology/WeatherManager

[Cervator]

Bump - particles didn't get picked for GSOC so it is open for regular work again. Pinging @Remi05 :-)

@viveret expressed an interest in this item, hoping one or both of you might be interested in working it outside GSOC. wc1997 too but unsure if there's a GitHub account there I can ping.

Willing to toss a good code bounty on here if it'll help somebody get motivated to take it on. Doesn't have to quite have the GSOC level of scope, but we could probably find some casual mentoring capacity to help it along

[Remi05]

Nice, I'll probably look into it during the summer but I'm quite busy with my finals at the moment.

[viveret]

@Cervator I'd do it following the GSoC schedule if possible, since that's what I planned.

[Cervator]

Sending it @viveret's way. @Remi05 you're more than welcome to look into helping out later if you find some spare time :-)

[Remi05]

Great!

[Cervator]

Linking http://forum.terasology.org/threads/general-particle-system.1277/ to here since that's the forum thread that went with @LinusVanElswijk's earlier attempt with a branch linked earlier in this issue.

The GSOC work period is drawing to a close within a month or so. Maybe as potential contributors and mentors free up somebody would be interested in picking up on this item again. We could do a code bounty of some substance for this one :-)

Edit: Added a token $25 bounty just as we had that in the team account on Bountysource. We can add a few hundred $ as needed for quality implementations, if it'll help get some work done :D

[Meta-Maxim]

Could I get this assigned to me please? Animated Particles 👍

[msteiger]

Hi Max, apprarently I cannot assign you until your first contribution. Go ahead and start - we will do that later.

[emanuele3d]

I'm thinking we should probably make an Epic of particle improvements.

[skaldarnar]

There you go @MaxBorsch 😃

We are waiting in suspense for your work on this! Btw, head over to our forums and leave an introduction there :wink:

[Meta-Maxim]

So I'm trying to salvage and revive all of Linus's work on this, because the way he did it is pretty much what I was aiming to do, and so I have a question on prefabs: How can I spawn two entities with one prefab, and have one of the entities reference the other in a field? Is there a way to make the entities separate prefabs and have one prefab reference the other? Here is what I need (emitter field is a EntityRef):

systemEntity {
  "emitter": emitterEntity
}

emitterEntity {

}

EDIT: It seems Linus only created particles with code using custom commands :| The system can't work with prefabs. I'll head down and write an introduction in a bit :)

[Meta-Maxim]

referring to the above post I figured the best way to handle the above problem was to merge system and emitter, since the system implementation could only have one emitter, so it might as well be the emitter and simplify the whole thing down for developers using particles. Essentially it's a change from Unreal E's method to Unity's method for particles. It's more important for particles to be prefab-compatible and intuitive than have just a tiny bit more control with systems that have multiple emitters for tiny performance improvements (Not even sure if multiple emitters would improve performance).

If anyone has thoughts on this I'd love to hear them before I move on with this system!

I still think that having prefabs be able to load/reference other prefabs is a good feature and should be added, so that we can have complex prefabs made up of several prefabs/entities.

[emanuele3d]

Hmmmm... there is a balance to be struck here.

On one hand you intend to work on this issue. We are grateful for this and even just simplifying the code to make more explicit what is a de-facto the way it currently works, such as particle systems being compatible with only one emitter anyway, makes sense.

On the other hand there's the longer term view. Architecturally particles have three main aspects:

• how do they start (the job of the emitter)
• how do they move (the job of fields and forces)
• how do they look (the job of the renderer)

The relationships between these aspects are best left as flexible as possible using composition, not for the sake of performance or simplicity but for what developers/artists can eventually achieve with them.

Multiple emitters should be able to inject particles in the same system so that fields and forces connected to that system only have to deal with one set of particles. Furthermore, the specific renderer for that effect would also only have one set of particles to deal with. This would also be beneficial if you want to pass particle data to the GPU: one big buffer with all the particles is much better than passing many small ones.

Conversely, it can happen that the similar sets forces and fields are used to affect different sets of particles. For example you might want smoke and embers raising from the same campfire. You could use separate emitters and of course separate renderers, but you might be able to apply at least some fields and forces to both.

If you want to go in this direction conceptually I'd be happy to assist you reviewing the code and advising as you go along.

Finally, in the film industry we used to say that particles do not look interesting unless one particle system is the emitter for a second particle system which in turn is the emitter for a third particle system and only the last particle system is actually visible. So, the capability of using a particle system as the emitter for another would be a bonus goal for you if you have time. :wink:

[portokaliu]

If I may add a request for improvement we have been running into some log spam issues when hosting a server with one or more clients on them. This spam was due to the particle entities being sent across the network so that the other side could interpret them as well.

Regarding this, changing the way the particles are sent would be a great improvement. Emitters would be the only ones that should be sent across the network, and even they should only be send under certain circumstances.

For example, if currently, a player hits a block, some particles appear and float outwards from said block. Just having the hit notification (which is sent) cross the network is enough to create a local spread of particles, so there would be no need to send each individual particle across the network. More so, not even the particle emitter should be sent, but rather created locally on hit.

Casting a spell might work the same way (a beam of particles from the player to a certain point). I'm not exactly sure how this would work per se, but sending all the particles across the network does seem like a hassle. Sending the particle emitter with all its params on the other hand would be the right idea.

If I'm getting any of this wrong please tell me :)

[Cervator]

And finally (Hi @MaxBorsch!) the idea of multiple entities per prefab has come up before, even if it may not be needed here anymore. I am unsure if it is included in @immortius' current work on extracting out the game's entity system into the standalone library Gestalt.

Welcome to the project and let me/us know if you'd like a Slack invite sometime, particles were flying around in there just earlier :)

[emanuele3d]

@portokaliu: thank you for bringing the network/multiplayer angle to discussion. I am not familiar with those aspects but what you are saying makes perfect sense: there is no need to send actual particle data across the network. What need to be replicated are just the entities that generate those particles.

That been said I can see a potential issue with this. Say there's an industrial chimney. They tend to have long, slow plumes. If the emitter starts only when the chimney enters the ViewingDistance relative to the player, every time the player will see the chimney starting to produce smoke. This might be reasonable in specific circumstances, but normally those chimneys are always on. So, either the particles need to get going some time before the player gets them in the visible range or the server needs to send some initial particle information.

Alternatively, particle effects will need to remain so small that the starting phase is effectively neglegible, but that's a pretty big limitation.

[portokaliu]

@emanuele3d Thinking about your example, I couldn't help but think of workaround or functionalities that would work with it :D

Unless the industrial chimney would be aimed diagonally (move outwards from the chunk it's in), as long as the chunk is loaded we can have the generator keep track of the time it was started. Following this, whenever we reload it, we generate the particles for the whole lifetime of said block (limited by the lifetime of the particle itself). Thus, a chimney having particles that last 10 seconds, we left it on all day while hunting/whatever, return at dawn and load the chimney chunk back in. The moment we load the chunk we see it's been active for 10 hours, and generate relevant particles. We get the least time necessary to generate particles (given their limited lifetime it will be 10 seconds), so we only generate 10 seconds worth of particle generation upon loading.

This of course still poses an issue for loading chunks from above the emitter, since the block itself wouldn't be in the chunk it would be generating particles in (this would be a common problem, not just networking issue)

If we move into the "stuff can go diagonally" particle idea, like if Goku suddenly appeared in our game and started shooting up mountains, we do need to replicate those particles without having a source (which would be a tad difficult since I'm talking about a stream of particles going across multiple chunks)

What could be done is have the chunk remember all particle streams that go through them at any moment. I have no idea how one would go about this, but it's pretty much the only way I could think of that has all chunks involved know there are particles.

Would that be a good approach?

[Meta-Maxim]

@Cervator A Slack invite would be nice, thanks :)

@portokaliu About the chimney problem, similar to how you describe the solution, Unreal Engine handles this by allowing you to give the system a warmupTime and when the system is loaded it quickly runs updates on it to simulate that time passing. That way it's up to the developer how they want it to work.

Why would you need to load the chunk above the chimney? Individual particles are not entities and are rendered separately from the chunk system (in the linus system). Why does every chunk need to know a beam is crossing into it?

@emanuele3d So... I'll change it to use a list of emitters. I also came up with a workaround for simplifying it if the developer wants their system to be prefab compatible (for one shot systems maybe?). Particle systems can have a selfEmitproperty, and when enabled, the system will look for and use an emitter attached to it's own entity. We also have to think about an efficient way to have "single-use" emitters that remove themselves after they emit while retaining the system. For example there can be a lot of dust from using a railgun, and it would be nice to have all of it under one system with multiple emitters, rather than spam create systems for each block.

[portokaliu]

@MaxBorsch If you descend from above the chimney and the chimney block would have a smoke stack reaching the chunk above it, if you only have the chunk above the chimney chunk loaded you will see no smoke (by what was said)

If you want other players to see particle beams and such without them having the chunk where the particle emitter starts (be it player or block) loaded, these chunks would not present any type of beam to the player.

On the note of particles, from what I've read each particle is and individual entity given a mesh and location component. If I've mistaken the code please do tell me where I'm wrong

[Meta-Maxim]

@portokaliu I see what you meant about the chimney. That is definitely not an easy problem to tackle :|

When I said particles are not individual entities I added "(in linus's system)", which is the new system we are talking about that Linus created and I am editing. In it, there are no individual particle entities or even objects, rather all the particles are stored in a ParticlePool, which contains arrays of all particle positions, lifetimes, etc. You can find his system here: https://github.com/LinusVanElswijk/Terasology/tree/feature/GeneralParticleSystems

[Cervator]

@MaxBorsch I need an email to send the Slack invite to :D Usually I just grab the email people use for forum registration but I don't think I see an account for you in there yet? Alternatively just get an email to me in some way, I'm not hard to find :-)

On particles yeah I've seen that problem in other games, at least one recently but which one escapes me (ARK? Every time I turn away from something then look at it again it starts producing particles from scratch). It is indeed tricky and affects plenty of other systems too. Imagine trying to simulate a river over multiple chunks, or very large power systems.

At present the magic dome in Light & Shadow is powered by an entity with a global sort of relevance so it always gets rendered, even if its chunk isn't near enough to a player to be loaded. But does it then never not render, even if way too far away to be visible? That's probably too extreme.

Again I find an excuse to trot out the sector thread - anybody tired of that yet? :D I wonder if we can do something for entities akin to Borders in world generation. An entity that has reason to believe it may need to poke into nearby chunks for relevance could register its intent to do so somehow and if something that could find it relevant (a local player) gets close enough then trigger some stuff. That could also even the playing ground between players using different view distances (a smoke stack registers for x chunks relevance, loads same time for a player on minimal view distance as one on far)

That could help avoid descending down toward a chimney and only getting new particles too late. Yet hopefully long-term not need the particle emitter to be globally relevant. It wouldn't do anything about particles not tracking from a player not looking right at them though, but I guess it could be an option to simulate whenever relevant, visible or not, yet only render when visible. Unsure how much of a performance impact that has.

For now I suggest just focusing on getting the particle system working better within current architecture, leaving the unloaded chunk situation for later (probably we should add this note to the Sector thread). The networking sync of particle emitter entities is something else - maybe there are shorter term fixes there.

Devil's advocate footnote to ignore as it just complicates stuff: imagine trying to fast-track a smoke trail from a large chimney - with a wind system enabled that affects where the smoke goes per tick. How far does the rabbit hole go ;-)

[Meta-Maxim]

@Cervator Funny story about my forum account, when I was signing up, I didn't realize how small of a community this was, and entered a silly name, expecting to see some "Username has been taken" message or a confirmation, next thing I know, my account name is "Developer" :P Sorry in advance for any confusion this causes.

As for the particles, a registry of subscribed systems like you described was exactly what I was thinking. Also should particle system entities persist? Is there a way to leave this up to the user? Where can I find general info on how persistance works?

[emanuele3d]

Hmmm, lots of issues here up for discussion but I think we are making good progress, at least conceptually. Let me comments on a few specifics:

@portokaliu said:

If you descend from above the chimney and the chimney block would have a smoke stack reaching the chunk above it, if you only have the chunk above the chimney chunk loaded you will see no smoke (by what was said)

Ok, above I broke down particle systems, from a functional perspective, into:

• how do they start (the job of the emitter)
• how do they move (the job of fields and forces)
• how do they look (the job of the renderer)

In practice however, from the perspective of the ES we probably need:

• emitter components
• fields/forces components
• particle look components
• particle data components
• particle system doing the processing

The particle data component is the key. That's what should hold everything together. Specifically it is an entity that should be associated with multiple chunks, based on the bounding box of its particle cloud. This way the particles are loaded as soon as one of the associated chunks is loaded. Crucially, if the particle data component also holds information on the emitters also the emitters can be loaded. This allow for the particles to continue without interruptions.

@MaxBorsch said:

When I said particles are not individual entities I added "(in linus's system)", which is the new system we are talking about that Linus created and I am editing. In it, there are no individual particle entities or even objects, rather all the particles are stored in a ParticlePool, which contains arrays of all particle positions, lifetimes, etc.

As far as I can see Linus system never got to the review stage. If it had this particular architecture might or might not have passed the review. For the way you describe it smoke particles would be stored alongside particles for water splashes, shooting stars, rocket exhausts and anything you fancy. That sounds like a one-size-fits-all that is unlikely to be efficient or expansion-friendly.

@MaxBorsch said:

We also have to think about an efficient way to have "single-use" emitters that remove themselves after they emit while retaining the system. For example there can be a lot of dust from using a railgun, and it would be nice to have all of it under one system with multiple emitters, rather than spam create systems for each block.

If you work within the ES an emitter component probably ends up having relatively few parameters to store and relatively limited processing methods. In this context I simply don't know if it'd be worth to recycle emitters rather than re-instantiating them. In the example you make recycling emitters sounds reasonable. In the five-pieces architecture I mentioned above (emitters, forces, look, data, system), it might be as simple as setting new xyz coordinates in the location component held by the entity also holding the emitter.

From a process perspective it would be good if you could describe Linus system's architecture and how it fits with Terasology's ES, i.e. its classes and how they interact together. I'm talking about plain English here, not code. Once we have that clarified (and ideally agreed upon) we can certainly look at code that initially doesn't even have to be functional. Just don't rush into implementing tons of features: the less code you send for review the quicker it can get approved!

Finally, @Cervator: given my interest in simulation "sectors" sound good. I'm just concerned it just kicks the bucket somewhere else. I am thinking, let's solve it once and for all: large but finite toroidal worlds and world-wide GPU-driven simulations. But let's discuss this elsewhere. ;)

[Meta-Maxim]

@emanuele3d Oh no you misunderstood Linus's system. Its broken down into the components you described pretty much exactly and organized, there is a particle pool for each system, different particles are not in the same pool.

[emanuele3d]

Ok, then I'm certainly looking forward to a PR about it. ;)

[Meta-Maxim]

Coming soon, just gotta get it working right with the new Tera and make sure multiple emitters works.

[Meta-Maxim]

Here's a quick overview of the particle system structure: Particle Diagram ^^ Please look before continuing to avoid confusion

Currently there are no force fields as we haven't really discussed them and I'm not sure how everyone wants them implemented.

Another thought I had: Force fields should have "target types" and/or a "layer", and particle systems decide whether they should be affected by certain force fields based on that. For example if you have a fan blowing, it would create a force field categorized as "wind", and smoke particle systems would react to it, while magic particles would ignore it because they specified to only be affected by force fields of type "magic". This would allow for realistic interactions between different particle systems / force fields between modules.

@emanuele3d Because statistics show you will be ready to reply and drop heavy knowledge on me anyways, I'll ask for your thoughts on this ;) 👍 / 👎

[emanuele3d]

Thank you for the diagram: it's very useful.

Some questions and notes:

1. What would be the responsibilities of the particle system manager?
2. Particle updaters: I suspect there should be a 1-to-1 relationship between updaters (probably inheriting from UpdateSubscriberSystem) and affectors components. I.e. a NewtonianForceUpdater would iterate over all entities containing a ParticleData component and one or more NewtonianForce components. The NF components would just store force vectors that the updater would sum up to calculate a resulting force. The updater would then proceed to update velocities and positions stored in ParticleData. Is this already the case?
3. ParticleRenderer: would it implement RenderSystem or does Linus/your system bypass this? In any case, is this meant to be a base class, to eventually have separate SmokeRenderer and WaterSplashesRenderer?
4. I'm finding the diagram somewhat ambiguous in its labeling of SystemEntity, EmitterEntity and so on. Are these specialized entity classes or are they just standard entities with particle-related components attached to them? If so I'd change the diagram to represent them visually containing components and then use a legend to describe them as entities.
5. What's the thinking behind multiple generators per emitter?
6. The descriptions of the affectors is quite mysterious to me. What do you mean with Force-Fields applied to an infinite field? And what do you mean with "they control how particles behave by default"? Who controls how they behave in all other circumstances?

Finally, the diagram would benefit from integrating some examples. For example in an affector component box you might write "Affector" (the class) on top, in bold, and then "gravity" (the instance) below in a normal font. It would add a more concrete meaning to the various objects. I'd also add boxes for LocationComponents, to see what might and might not have a position.

Looking forward to your reply.

[Meta-Maxim]

Thanks for the suggestions @emanuele3d .

1. Particle System has a little red System label next to it to signify it's an "actual system"(with @RegisterSystem) as defined by the wiki here. Its job pretty much exactly fits the description of a system on the wiki, it handles events from particle system components (which are not "actual systems" with @RegisterSystem), such as adding/removing them, and calls the update loops of the ParticleSystemUpdater. It also keeps the registry of affector and generator functions (more detail following).

2. So I think what you're calling an "updater" is what Linus called a "affector function." Essentially, an affector function is an object that just holds a function that takes a certain type of component (1-to-1 like you described) and a particledata, and uses the data in that component to set the fields of the particle every update. So for each affector component, there is an affector function that uses it as input to determine particle fields during updates.

3. ParticleRender is abstract. It is a base class and does not extend anything, but the subclasses are very broad renderers, in fact there is only one, DisplayListParticleRenderer. It just renders a particlepool using GL lists. Currently there is one renderer for the whole entire manager (so all systems use it). If we want multiple renderers, that should be an easy addition, just give systems a field specifying what renderer they wants to use and keep a registry of available renderers.

4. Yeah they're all regular entities containing components, will fix.

5. A generator component is like an affector component, it just holds data for a (1-to-1) generator function to use to set fields. For example to randomize scale 1-5, emitter.addComponent(ScaleRangeGeneratorComponent (1, 5)) later when a particle from this emitter is created, a ScaleRangeGeneratorFunction.onEmission(scaleRangeComponent, particleData) is called to set the field based on the data in that component. Same idea as affectors/your updaters, just for generating instead of updating.

6. Sorry, confusing example... Hopefully cleared it up earlier, I'll update the diagram.

After all that reading, here's some eye candy from the Linus/Me system.

[Meta-Maxim]

Finished up adding support for multiple emitters per system. The updater now balances out the pool so that each emitter gets a partition of the system's particles. Here's an example image below, the red outlines show systems, the blue circles are emitters. In the system with only one emitter, the emitter is able to use all 500 of the particles in the system. In the system with lots of emitters, there is still only 500 particles available, so each emitter emits a share of the particles (the explosions are hardly visible in the second system because there's a few more emitters off-screen, so each emitter only gets about 40 particles).

I also updated the diagram.

I think I'm about ready for a PR, I'll tidy up tomorrow.

[Cervator]

Oooooohhhhhh .... shiny! Or explodey. Looking forward to it! :-)

Sketch looks cool, but as a mere mortal I'm just curious if I can attach a single component to an existing entity and tadaa particles? So if you wanted a gelatinous cube to be sparkly just attach SparklyComponent after having initialized it with a bit of config?

[Meta-Maxim]

@Cervator Currently you can do that with two components, there always has to be a ParticleSystemComponent and at least one ParticleEmitterComponent. Nothing is preventing you from attaching both to the same entity.

To make it more comfy, I think what I'll do next is create a ParticleFactory to handle all the system/emitter nonsense and allow mortals to retain their sanity. So you can just pass an entity (gelatinous cube), a texture, a list of whatever generators/forces you want, and tadaa... burning gelatinous cube.

[emanuele3d]

Thank you @MaxBorsch for the additional clarifications.

Alright then. Let's see the code. Looking forward to it!

[Cervator]

Closing this as tentatively completed - there are still some more tweaks to add and documentation to do, but effectively this issue is covered :-)

See #2909 for some follow-ups and #2999 for a tutorial module request.