search

junkdog / artemis-odb

A continuation of the popular Artemis ECS framework
BSD 2-Clause "Simplified" License
776 stars 111 forks source link

(Extend API for) Entity DSL, OnReplace & Pooling #348

Closed gjroelofs closed 6 years ago

gjroelofs commented 9 years ago

This issue comprises 2 aspects:

The combination of syntax which provides easy access to mutation of an entity, "onReplace" subscription and pooling of components will allow a coding style similar to Functional Reactive Programming. It will also allow a faux-immutable style of programming while allowing the coder to sidestep the performance overhead of this style when needed.

Concretely the following advantages:

E.g.: Given a Health and Mana component, both accepting a float value.

Entity e = world.createWorld();
e.addHealth(10)
  .addMana(10);

e.replaceMana(20);

e.removeMana();

Mana manaComponent = e.getMana();
// Keep track of a parent / child relationship. Where Parent & Child are both components containing an Entity variable named "value". 
world.onComponentChange(Parent.class, (e, oldParent, newParent) => {
     newParent.value.addChild(e);
});

One approach could be to use a code generation library which allows proper parsing of Java code which supports Maven & Gradle integration. And example of such a framework: http://spoon.gforge.inria.fr/examples.html

The basic strategy for code generation could be:

DaanVanYperen commented 9 years ago

Some observations.

gjroelofs commented 9 years ago

My comments, the points I do not comment on I agree with in general.

DaanVanYperen commented 9 years ago

State changes should include component creation/destruction. Could you give some examples of this? These should be covered by Add & Remove Subscription listeners already provided. Or do you mean triggers on creation/destruction from Pool?

Just making it explicit, couldn't tell if the listeners will trigger in cases of a newly added component (and primary use case) upon removal from an entity.

and

? :D

I think using well-defined build systems will be preferable over plugins we need to maintain.

I agree, the convenience/utility drops a lot though if we can't have auto completion work out of the box. At least for my jam use cases .;)

Namek commented 9 years ago

About syntax: what about https://github.com/junkdog/artemis-odb/wiki/EntityFactory ? Isn't extending this just enough?

junkdog commented 9 years ago

We'd need to have component deltas opt-in, not affecting overhead when not in use. Feels like the most straightforward way would be to wrap entityId in another generated class, the "EntityWithComponentMethods". Most methods which take Entity as a parameter already have int overloads).

Refactoring component storage is more problematic though.

@Namek EntityFactories are too verbose and are problematic to set up.

gjroelofs commented 9 years ago

@DaanVanYperen

I'm suggesting to generate a class in a task/precompile step that needs to run after changes to components. Autocompletion and the like works because of the generated class which is available after the task is run. (which ideally would be linked into an IDE trigger)

The "and" was meant to describe I would address the issue with the following answer :)

@Namek, @junkdog I see two options:

@junkdog

Could you detail the refactoring required for component storage a bit?

gjroelofs commented 9 years ago

@junkdog

I'm a bit confused on the wrapping of entityId. My line of thought was to generate the componentID, and bypass ComponentMappers.

gjroelofs commented 9 years ago

After some work with Spoon; the naive implementation only scans the code in the current project. I'm now looking whether it can be configured to look into dependencies as well.

Easy Route: Manual specification

Looking at it from a different perspective: To implement proper component ID generation we would need some mechanism that shows which Components are used by which World. If this is mechanism is written manually in the consuming project we can catch all Components there. The obvious downside being that you need to manually specify which components will be used by the World.

@junkdog

Food for thought

The constraint that the number of Components is not known at instantiation of the World leads to some slight overhead (ComponentManager; bound checking, expanding Bag). What we could do is create a common interface for the current ComponentManager, extend it and create one that uses integrates the fact that we know which Components are used at instantiation.

Hard Route: Search

Most uses cases for a World have a single instantiation location where all Systems etc are added. We could go on a recursive search through all used systems and their dependencies to find which Components are (in)directly used by a World.

gjroelofs commented 9 years ago

Alternatively, we can use the concept of a Class that contains all known ComponentMappers which are instantiated lazily per World.

The EntityDSL builder implementation could lookup and delegate to this class upon instantiation. (Or hide the ComponentMappers internally)

junkdog commented 9 years ago

Ok, a bit more awake today (tried to process this yesterday - and other things, but I was beyond salvation).

I'm suggesting to generate a class in a task/precompile step that needs to run after changes to components. Autocompletion and the like works because of the generated class which is available after the task is run. (which ideally would be linked into an IDE trigger)

Refactoring would break without IDE integration however. It could be added as a later step, but I think that at least intellij should support it (from my understanding, intellij plugings are vastly more easy to write compared to eclipse plugins, and eclipse's popularity is fading).

If only we had roslyn on the JVM and plural IDE side... I thought truffle/graal was going to be something towards that end, but it's not as far as I understand it(?).

Could you detail the refactoring required for component storage a bit?

I missed the constraints part ("Users which do not use the code-generation should be able to use Artemis as-is."). I was mostly thinking about saving the previous component state; it kinda lends itself to "double buffering" of two arrays (ByteBuffers or such) per component type. *But then I started thinking about References to components so disregard my remark).

Most uses cases for a World have a single instantiation location where all Systems etc are added. We could go on a recursive search through all used systems and their dependencies to find which Components are (in)directly used by a World.

ecs-matrix already scans for artemis types. It'd need to be extended to scan dependencies too though.

Shadow Entity.java in the consuming project

It's more hackish, but I think brevity wins - there's no way around it being invasive somehow; since it's opt-in and with an eye on readability, shadowing Entity means less noise in project code.

References to components

Stashing away references to components is disallowed, if we're replacing and keeping previous component state? https://github.com/junkdog/artemis-odb/issues/269 could guard against storing actual components in bag, arrays etc, but dealing with storing a reference to a field is much trickier:

public class Position extends Component {
    // user might want to retain xy elsewhere
    public Vector2 xy = new Vector2();
}

We could use the same instance for the component though; copy the actual instance's data to the previousState instance.

DaanVanYperen commented 9 years ago

I'm a big proponent of introducing better state change events, love that in core. The composite mapper/mixin entity precompilation however is something that is pretty fundamental pivot in how the framework operates. If it's optional it does not belong in core imo.

I'm suggesting to generate a class in a task/precompile step that needs to run after changes to components. Autocompletion and the like works because of the generated class which is available after the task is run. (which ideally would be linked into an IDE trigger)

Understood. This does shift us further towards the brittle end of the spectrum and I fear the paradigm shift would cause some distrust in the typical user if we put it in core. It would be a good plugin for users who explicitly choose to offset the unavoidable IDE glitches in exchange for the speedup on the other end.

There's a lot of dials and knobs being added to odb causing the exposed api to grow immensely and making it harder and harder understand and maintain. DSL vs rainbow flavors of JSON vs Factories vs Editors vs Proxies vs Archetypes vs code generation. Before this implementation specific discussion we need to set out some clear goals for core and what we want to consolidate/replace/externalize.

DaanVanYperen commented 9 years ago

Perhaps we can steer this discussion towards opening up the API to allow more invasive richer plugins and deal with that how we deal with all the contrib stuff; let the popularity dictate what we include in core.

gjroelofs commented 9 years ago

I'm also a fan of keeping core light, and introducing this in a plugin.

The added benefit of shadowing Entity.java is that we can keep the API we need to access protected, without exposing it. (Or we could just generate into "com.artemis")

I was actually surprised that serialization and the like was introduced into core. From our end we see that a generalized serialization implementation tends to run into those special cases which need proper serialization either way. (Or we don't serialize the World; but a parameters object that sets it up. Most of the time we are dealing with hidden state (UI etc) that Entity serialization doesn't handle)

I already discussed my thoughts on streamlining general design with Junkdog and am in favor of opening the discussion on that. (Although I'm also skeptical of the practicality of it.)

junkdog commented 9 years ago

I was actually surprised that serialization and the like was introduced into core.

Corner-ish case, but it's a useful USP + helps establish some measure of "idiomatic artemis"; in this case - *Resolver managers which allocate transient components from asset reference components.

Most of the time we are dealing with hidden state (UI etc) that Entity serialization doesn't handle)

You can extend SaveFileFormat and add pojo:s holding additional data (or write serializers for systems). It does kinda assume that World is central to the overall design though.

(Although I'm also skeptical of the practicality of it.)

Skeptics rojoice!

gjroelofs commented 9 years ago

Corner-ish case, but it's a useful USP + helps establish some measure of "idiomatic artemis"; in this case - *Resolver managers which allocate transient components from asset reference components.

I can get behind that; one of the first decisions in our framework was the introduction of AssetReferences and it's a concept not that many devs use while it can alleviate a lot of headaches. (even not in ECS)

You can extend SaveFileFormat and add pojo:s holding additional data (or write serializers for systems). It does kinda assume that World is central to the overall design though.

We found that in a lot of use cases we didn't need to serialize the entire world and primarily on focused on well-defined entry points that act as both initialization and reset. Starting a level and loading a level are the same thing; which removes some of the boiler plate that normally needs to be written if you serialize the entire world state (i.e.: a full world save file format isn't easily setup from code)

Skeptics rojoice!

Do you mean that you would open the discussion up for things such as Entity -> Group -> System, and listeners at each level?

junkdog commented 9 years ago

We found that in a lot of use cases we didn't need to serialize the entire world and primarily on focused on well-defined entry points that act as both initialization and reset.

Hmm, I think our goals converge, but approach/implementation differs. I tend to serialize quite frequently and in different contexts, and it's pretty much a unified approach.

Since loading is additive, we use three serialized states/save in our current project:

theme is pretty much RO, dynamic and save are... saved. The only custom code we use is populating the state pojo.

Our Prefab class just wraps a tiny save (1-10 entities).

Current game has the editor is built straight into the game; so any dangerous operations record a snapshot of the world prior to executing, in case of failure, the FailsafeInvocationStrategy rolls back until it finds a working state (meaning, world.process() doesn't throw). (I took the idea from unity actually, but multiple states instead of just pause-play + recoverability).

There's some overhead involved with keeping the world always serializable, but it's usually pretty easy to fix. I do think it helps in uncovering bugs too, save-load is bound to venture into some unforeseen code paths with a shady state.

Do you mean that you would open the discussion up for things such as Entity -> Group -> System, and listeners at each level?

Each level? Crazy talk!

I'm starting to think that this is really two different tasks: code gen and jacking in more granular listeners.

It'd be interesting to experiment with code gen, but in the end, I don't think we can provide an easy-to-use/easy-to-setup solution. Brittleness is another thing I worry about. Xtend feels like the overall easiest approach.

Listeners are probably more feasible. Could open up ComponentManager a bit I suppose, bnt it's something of a pandora's box. It might be the equivalent of sun.misc classes, though they're pretty stable atm. Requires some tinkering before it becomes clear what needs to be opened up.

Component deltas is probably the biggest challenge, and how it affects performance.

Per-component pools, wouldn't that interfere with multi-world scenarios, or pay the lookup cost? Unless it's a world-bound poolId. (nvm)

The faux-immutable style of coding could lead to problems in multithreading

artemis-odb will most likely never be multithreaded; it'd require profiling/introspective tooling to trace why code runs 3 times slower with 8 cores.

gjroelofs commented 9 years ago

I've dug a bit further in Spoon vs. Xtend:

For the Entity DSL approach: An approach which completely sidesteps hacking with component IDs is to generate an Entity which has stubs for the various add/replace/remove methods; which are then overloaded by a world-specific Entity. This class then has static ComponentMappers to which the methods delegate. A specific EntityManager which instantiates these specific Entities are then generated and used by each world.

This approach could be used in subdependencies as the signature matching will hold up in the overloaded projects. It's highly unconventional though :D

gjroelofs commented 9 years ago

Approach:

Alterations:

Problem:

DaanVanYperen commented 9 years ago

@junkdog think we should strive to get contract changes into 1.0.0 release, specifically for:

Systems depend on Entity.flyweight(), which instantiates the wrong Entity type.

by moving it to EntityManager, and perhaps (@junkdog any performance risks?)

Introduction of EntityManager in WorldConfiguration Remove final from EntityManager, Entity, RecyclingEntityFactory, expose for extension.

DaanVanYperen commented 9 years ago

API changes are in, pop a new ticket if you need any more changes on odb side!

gjroelofs commented 8 years ago

Generation hook strategies that are possible:

Ideal #3: Java Parser that parses all .java files, generate Xtend Annotation that hooks into #1

    #1 Xtend Annotation: predefine the components that need to be generated, generate Xtend, delegate to 
                        Xtend compiler which generates Java.
        + Code doesn't need to be in compiling state.
        + Plugin clean as it uses Xtend API
        - Xtend class shadowing does not work yet*
        - Xtend class define required
        - Manually define which Components to use

    #2 Xtend Annotation: predefine the components that need to be generated, generate Java
        + Code doesn't need to be in compiling state.
        - Xtend class define required
        - Plugin code could be messy
        - Manually define which Components to use

    #3 Java Parser scan for Annotation, generate Xtend that hooks into #1.
        + Code doesn't need to be in compiling state. (? if we get Java AST to work)
        - Manually define which Components to use

    #4 Build plugin that generates code after compilation.
        - Doesn't work if code isn't compiling
        + No manual definition of components
DaanVanYperen commented 8 years ago

4 is definitely less cool but sounds less fragile. What was your stability experience playing around with xtend?

As it relates to this discussion, roadmap for 1.0.0 we've settled on the following future for the pre-existing component related API.

So all in all, we're still looking for a concise way of assembling entities. ComponentMapper is too verbose. Same with EntityEdit, EntityBuilder isn't pooling compatible,

Factories might work if we make them a bit more flexible and transparent. Still, @junkdog found the annotation processor to flake a bit, so can't say our path is clear.

gjroelofs commented 8 years ago

Fair enough. The Xtend compilation cycle worked quite well if it weren't for the overloading issue (which I've noted upstream and won't be addressed until the next release).

Everything works so far if I stick to Xtend.

It's actually far less fragile due to it working if the project doesn't compile; but yeah.

DaanVanYperen commented 8 years ago

Didn't mean to imply DSL can't fill that hole, it might.

wouldn't consider a build step that doesn't run due to a previous build step failing fragile, just having a predictable drawback

Then again, read 'build plugin' as a maven/gradle plugin that is manually run, your reply implies something broader perhaps?

junkdog commented 8 years ago

(this one stays as a historical document)

/housekeeping

DaanVanYperen commented 6 years ago

Putting historical documents in the freezer to help out grooming.