Component and Entity pooling

[apotapov]

I seem to have trouble posting on the Artemis forum, so I figured I'd ping you here instead. I'm the guy working on gdx-artemis. (https://github.com/apotapov/gdx-artemis)

You guys have done an awesome job cleaning up Artemis, and it seems our intentions are pretty well aligned: improving the implementation and making it more peformant, specifically in memory management. Ultimately, I would like to see our codebases converge, rather than diverge maintaining separate forks.

I'm not particularly married to the idea of having libgdx as a dependency for artemis. As a matter of fact in hindsight it's probably a bad idea to do so.

My biggest issue with Artemis has been the lack of entity and component pooling. It wouldn't be so bad if I could instantiate my own Entity and Component managers into an Artemis world, but the framework seems to be hell bent on preventing any reasonable extension of classes. (almost everything internal is either private or package protected)

So, long story short, my proposal is to work with you to add pooling to Artemis. Thoughts?

[lopho]

Hi there! I think we've already met in a commit :).

I think it's a goodthing™ to converge, as you put it.

To the managers: This would be great, as long as the API stays stable. So if you can code it out without breaking the current public facing stuff, I encourage you to do so, but please set up some unit tests to ensure.

I wonder what @junkdog has to say to your proposal, as it's his repo.

Anyways, great thing, always hoped to have a new upstream!

[junkdog]

Ultimately, I would like to see our codebases converge, rather than diverge maintaining separate forks.

+1

Sorry, been a little busy with work and off-work activities the past couple of days. I'll get back to this thread once I'm done with today's work :)

[junkdog]

I think pooling entities and components is a good idea. I feel now that artemis 0.4.0 has been released - in my mind meaning that we've fixed all known issues etc (we'll see how long that lasts) - it might be time to introduce some new functionality into artemis. So far, I've tried to limit the API changes to a minimum to ensure that adoption would be as smooth as possible.

But... back on track: I've been pondering how to do packed components (#9): my approach would basically involve a couple of new methods in world for creating components (meaning, ask the World to "instantiate" components for you). This is how I guess you envision component reuse too? Or how do you intend to track components so they can be retrieved from the cached pool? One downsize, but a reasonable compromise imo, is that it would require a zero-argument constructor for components, and no final fields. This could ofc be validated by the IDE if we code a little annotation processor to verify that the components are valid.

Entities can be pooled pretty easily, if I remember correctly.

[gjroelofs]

Entity pooling should be pretty easy; the abstraction is already there in EntityManager.

I'm also working on a fork of artemis, and we choose to go a route in which message passing is the main form of interaction between systems.

The main change is essentially that an EntitySystem now has:

void begin(Event e); void process(Event e); void end(Event e); void processEntities(Event e, ImmutableBag entities); boolean checkProcessing(Event e);

Systems are registered to the Class of an Event. Event is a tagging interface. Artemis has a DeltaTimeEvent implementation to cover the normal "process" logic. The SystemEvent(Class) is a special type of event for which a System is by default registered for.

I've also wrapped all types of Input as Events; to ensure that any input is handled in the World thread instead of the OpenGL thread.

Some other alterations: ) DisabledSystemComponent allows entities to be disabled for specific systems. ) We've removed Entity.changedInWorld; Entity.addToWorld. The changedInWorld can already be intercepted upon adding/removing Components in Entity itself, and the creation obviously happens through World.createEntity. IMHO; the extra calls only get forgotten and lead to hard to diagnose bugs. *) We've altered the Manager injection to work with classes that are not leaf nodes in the class hierarchy. (as in, superclasses also get succesfull injection)

[apotapov]

@junkdog The way I implemented pooling in gdx-artemis is pretty straight forward. There is a member function on the World for creating components of certain type:

    public <T extends Component> T createComponent(Class<T> type) {
        return cm.createComponent(type);
    }

And then in the component manager we have:

    public <T extends Component> T createComponent(Class<T> type) {
        return Pools.obtain(type);
    }
...
   protected void removeComponent(int entityId, int componentClassIndex) {
        Array<Component> components = componentsByType.get(componentClassIndex);
        if (components != null) {
            Component compoment = components.get(entityId);
            if (compoment != null) {
                components.set(entityId, null);
                Pools.free(compoment);
            }
        }
    }

Entity is similar.

I couldn't figure out a way to force default constructor on the components. But I figured the first time the component is used, the obtain call will blow up so it's relatively safe. Although might not be as straightforward to developers not familiar with Class.newInstance(). As for final fields, that should be okay as long as they are instantiated inside the default constructor.

I also changed the Component to be an interface rather than a class. I feel like that's more appropriate. I added the Poolable interface to the component, more as a reminder to the users of library that the components are pooled and to reset the state appropriately.

The way i envision changing this non obtrusively is to create an inteface IComponent or some such, get the Component class to implement it and deprecate the Component class. That should make it an easy transition.

@methius I like the idea of event driven architecture. I think it's a cleaner approach than tightly coupling systems together. One thing I'm having difficulty seeing is how to make the message system generic without having to resort to "instanceof" and casting for different event types. Also, how do you manage subscription to the different types of events.

Perhaps this can be implemented with some sort of a message queue system (ala RabbitMQ). I would err on the side of creating a new set of EntitySystems for this purpose though. We shouldn't force the users to implement event handling if they don't need it.

[lopho]

@methius I'm not not so sure on tacking event handling onto artemis. My view of an ES has always been a focus data (e.g. get data, set data, transform data), not a game engine for handling anything else.

That out of the way, may I ask you, what these events are? You mention input. But what about data that influences other data, without any user interaction? And what about other usages that aren't necessarily revolving around user interaction? How are events emitted by systems, would you use listeners to keep track of components, to trigger events once something is happening? Would that not require to still iterate through all entities with given components (though not by the system, but by the listeners)?

I not trying to keep you from it, just curious how this would play out.

changedInWorld, addToWorld: would that trigger every time you add a component? Currently it is only triggered once when you call changedInWorld, no matter how many components are being added. This is the biggest advantage I see in keeping that call.

Lots of questions :D. I hope I'm not rushing you.

Anyways, great to have more people on board.

[lopho]

@apotapov : Nice, a simple kind of component pooling. Sounds reasonable (didn't look at every detail yet). What is the Array class though? I guess a dynamic array? (libGDX? Never used libGDX, should try it soon.)

[apotapov]

Yeah, sorry, I should have made that clearer. I replaced Bag in gdx-artemis with an Array class from libgdx. It's basically a dynamic array that's memory and execution optimized. But yea, the code above just illustrates the idea of Pooling components.

As for messaging, perhaps something like Guava's EventBus could be a nice solution. (https://code.google.com/p/guava-libraries/wiki/EventBusExplained)

We should probably start a separate thread to discuss event processing in artemis though. I think there's a lot of potential there. One thing that pops into my mind already is event pooling. (seems to be a recurring theme :)

The biggest win would be the decoupling of Systems from each other. For instance, in the current game I'm developing, I have a system that's basically an Entity Factory. It creates Entities of various types and injects them into the world at the request of other systems. Currently, all the systems that need an entity created need to have an instance of the EntityFactorySystem in them and call the appropriate method to create a specific entity. With event based system, you just kick off an even to create an object, and that is handled for you. At that point the EntityFactory doesn't even need to be a System. (I made it a system for discoverability: world.getSystem(EntityFactorySystem.class)

This can also apply to playing sounds, processing user input, and a number of different functions. As I mentioned previously, I would be cautious about roping this type of functionality into EntitySystem, but it probably makes a lot of sense as a child class.

[gjroelofs]

@lopho ChangedInWorld, AddToWorld: the ChangedInWorld is queued internally by a simple boolean that indicates whether the entity is "dirty" (Aka should trigger a change call the next process or not), which alleviates the performance issues with repeated change calls. The time spent solving the absence of the bugs far outweighs the trivial overhead of the boolean check :-)

@apotapov Our EntityFactory uses static methods for access, and acts much like the AssetManager in LibGDX. We've also opted for the choice of EntityTemplates, which have a TemplateBase, and a TemplateVariant. The variant is applied on top of the Base to provide variations; possibly with ranges for values.

I've looked at both Guava and RabbitMQ and decided that both have significant overhead in either code verbosity (RabbitMQ); or performance, per example due to the usage of the reflection API (Guava). Not too mention that the overusage of reflection in APIs which are not in-house will give problems during LibGDX compilement to platforms that do not support it.

Our current implementation uses simple instanceof checks (trivial performance impact, at the cost of some code verbosity). Most of the system only respond to a single type of event, so the check is not even necessary.

I am however working on seeing whether a custom made Guava-like implementation will provide any benefits. (The main objection being that you do not have a central entry point into your EntitySystem, ala begin(), process(), end().)

Event Usecases Currently we use events for things like:

• InputEvent -> KeyEvent, TouchEvent, ControllerEvent
• DeltaTimeEvent
• MovementEvent -> DiscreteMovementEvent (Graph/Matrix), ContinuousMoveEvent
• PauseGameEvent
• EntityEvent -> EntityAbilityEvent, EntityAttributeEvent, EntityEffectEvent, EntityModifierEvent, EntitySelectedEvent
• GlobalSoundEvent
• TriggerEvent -> EntityTriggerEvent, SimpleTriggerEvent, UITriggerEvent (Used for state machines, all of our UI resides in a FSM, in extensions of a FSMSystem)
• CameraEvent -> CameraPositionEvent, CameraZoomEvent

Pooling I tend to avoid pooling until it becomes obvious that it will have a performance benefit. Bugs tend to arise with the boiler plate code that is required for pooling to work across subclassing. (Reset() / Set()) My current solution will probably be to extend Lombok to handle creating the copy constructor and reset() logic.

[gjroelofs]

Also, Systems are registered for specific classes of Events at insertion into the World. So they can either register for the specific subclass, or generic superclasses. Before they receive the event in the begin()/process()/end() cycle, checkProcessing(Event) is called to see whether the System accepts the event. (Which has a default return true implementation)

[junkdog]

Too much text to digress atm (I'm running out of time) - interesting stuff nevertheless, have a few questions, will ask/reply here tomorrow :)

[apotapov]

I'll be honest, I'm a little obsessed with memory management. I started my career as a C++ developer and it's been forever burned into my mind. Moving to back to java was a relief as memory became less of a concern. But now switching back to Android I can feel my old synapses firing again.

Interesting approach on the event based system. I guess casting is a simple way to avoid the issue.

I guess I didn't think too much about guava's abuse of reflection as a potential performance hit. Again, as with memory, seems like a lot of it is premature optimization though. I think for myself I'll give guava a shot and optimize if needed.

I started a new issue for adding an event system to Artemis: #17. Let's move the discussion there.

[junkdog]

• Entity pooling implmented
• Component pooling WIP (but working) in #25