Integrating dustArtemis into the benchmarks

[dustContributor]

Hi! Well, I'd like to contribute the integration of dustArtemis into the benchmarks. Possibly mirroring the repo in GitHub along the way, I'm still not quite decided.

In any case, what steps should I take to do such thing? "Mavenize" the project?

[junkdog]

Hey!

Ideally, publish the most recent dustArtemis artifacts/jars to maven's repositories; this - in addition to being a convenience for the benchmarks - also makes it easier for others to utilize dustArtemis (especially now that gradle is the default setup for libgdx projects); maven and gradle both use maven's repositories to download dependencies.

Easiest is probably to copy the artemis-0.4.0 folder (or artemis-0.6.5, if you have more component types), change the artemis-odb dependency to point to dustArtemis and resolve any errors.

I only benchmark released versions, but if you're between releases, we could use a snapshot branch (it's what I did before releasing 0.7.0).

setting up a sonatype account for publishing maven artifacts: http://central.sonatype.org/pages/ossrh-guide.html

[dustContributor]

also makes it easier for others to utilize dustArtemis

You and I apparently don't agree on the definition of "easier" :smile:

I only benchmark released versions

I don't have releases (although I set one in the repo), I however don't do commits that I know they're breaking things, not in the master branch at least.

All right, already cloned Bitbucket repo on Github. Now onto that Maven repository thingy...

[DaanVanYperen]

Good to see some java 8 pioneering going on with Artemis.

Wanted to resonate this point with @junkdog, concrete releases and published artifacts are health and maturity indicators for a project, (Unit tests for libraries too). Extra effort on your part, but with a payoff in users.

[junkdog]

You and I apparently don't agree on the definition of "easier"

The benefit of using a build/project tool with automatically resolvable dependencies is that you're not tying the user down to using a specific IDE. I have no idea about the proportions, but there's a pretty big chunk of people using intellij; netbeans is probably on the lower end of the scale, while Eclipse still holds the majority in its grip.

Also, hinted above, having a central repo where all code is downloaded keeps the repositories real small, as there aren't any binaries to keep track of.

Additionaly, whenever using a CI server, you need to configure the project in an IDE agnostic format.

I however don't do commits that I know they're breaking things, not in the master branch at least.

We never do, but we break stuff anyway! :smiling_imp:

I thought I had all bugs from vanilla artemis sorted out by 0.4.0 (October 2013), but 0.7.0 still fixed a few lingering ones.

(Unit tests for libraries too)

(Psst, grab artemis-odb - or write your own; makes refactoring and life overall easier, in the loooong run)

[dustContributor]

Good to see some java 8 pioneering going on with Artemis.

I really like it! Say, having entities processed in parallel is as simple as calling "parallelStream" on the 'actives' Bag when processing the system (of course, minding side effects and such).

The benefit of using a build/project tool with automatically resolvable dependencies is that you're not tying the user down to using a specific IDE

At least the way I see it, or what I intend to do with dustArtemis, its for it to have no dependencies, maven in itself its a dependency too. I want to keep it relatively simple.

Also, Artemis has many, many places where just plain download a .jar and linking to it won't cut it. For example, all ComponentManager's bag sizes. If you intend to make the game startup as smooth as possible, first thing I'd do is set all sizes to a reasonable estimate of what the game will use, so to avoid any 'grow' calls and array trashing.

I don't intend to provide a drop in solution to whatever game you have, rather as a set of classes you can integrate and tweak to your desires. Which is basically what I do.

In any case, its not that I don't see the benefit of dependency resolvers, there are many big libraries and frameworks out there that are built upon the shoulders of many other libs. Or simply very modular setups where you can tweak the build process and get a different outcome for the same library. I do believe that for small libraries with permissive licences, it might be more of a hassle dealing with that rather than just cloning the repo and integrating the project to whatever IDE you're using.

I thought I had all bugs from vanilla artemis sorted out by 0.4.0 (October 2013), but 0.7.0 still fixed a few lingering ones.

I call them features.

We never do, but we break stuff anyway!

Once I was tampering with the removeFromSystem() method, forgot to clear the system bit from the entity bits. Null pointers fun times were had.

[junkdog]

At least the way I see it, or what I intend to do with dustArtemis, its for it to have no dependencies, maven in itself its a dependency too. I want to keep it relatively simple.

You can still ship it with the eclipse project files - you don't need to force it down people's throats; but intellij and netbeans have native maven integration. This is the path of least resistance for people who want to make contributions or their own custom modifications (ofc, I opted out of supplying eclipse IDE files, but hypocrisy aside...).

If you intend to make the game startup as smooth as possible, first thing I'd do is set all sizes to a reasonable estimate of what the game will use, so to avoid any 'grow' calls and array trashing.

It'd be more flexible having such things as parameters passed during initialization; no need to change the underlying source.

I do believe that for small libraries with permissive licences, it might be more of a hassle dealing with that rather than just cloning the repo and integrating the project to whatever IDE you're using.

My primary reason for not using universal tween engine anymore is that it's not available in any maven repos.

For example, if you're developing your game with maven, to add say artemis-odb to your project, you only add a few lines to your pom.xml:

<dependency>
    <groupId>net.onedaybeard.artemis</groupId>
    <artifactId>artemis-odb</artifactId>
    <version>0.7.1</version>
</dependency>

With gradle, it's a simple one-liner:

'net.onedaybeard.artemis:artemis-odb:0.7.1'

In addition to having the library automatically downloaded, you can configure your IDE to always download the javadoc+source for each dependency too: as such, when debugging or just using the API in general, the sourcecode and documentaion is right there. No need to download and link separate source + javadoc jars.

With libgdx now defaulting to gradle projects, which resolves dependencies through maven's repos, you'd be cutting off a big chunk of potential users - simply because us programmers are lazy people. The libgdx userbase make out the biggest demographic for java gamedev library developers - by a wide margin - and there are a lot of ECS libraries out there. As libgdx recently embraced ashley as one of its own, there's very little incentive for people to go out and look for a different implementation in the wild.

[dustContributor]

You can still ship it with the eclipse project files - you don't need to force it down people's throats;

You misunderstand, I don't care for those. I just have them because of laziness. The idea would be just to clone the repo and convert to Java project, no matter whatever IDE you have, that's usually a very simple thing to do, at least with self contained libraries (ie, git clone, add to workspace/work group/projects, set as java project, done).

Hell, I don't think it would compile even if you clone it from Eclipse. That's why I suggest cloning the repo and converting to Java project (in whatever IDE you use).

It'd be more flexible having such things as parameters passed during initialization; no need to change the underlying source.

Sure, library should be configurable (that's actually what I was implying), and it will be. For the meantime, having the sources around is the next best thing.

No need to download and link separate source + javadoc jars.

I'd argue that cloning a repo, as I said, for a small self contained lib, has the same benefits. Saying "This project depends on this other project" is rather simple in modern IDEs too.

Now for a bigger lib, like your own artemis-odb or libGDX (specially since it has various modules/extensions), I do think having some form of dependency management is a good idea. Linking the whole .jar + source + javadoc on libGDX is a pain. NiftyGUI also comes to mind. I'm not arguing on that front.

My primary reason for not using universal tween engine anymore is that it's not available in any maven repos.

Sweet! I didn't knew about that lib, thanks for mention it!

With gradle, it's a simple one-liner:

I've been eyeing gradle for a while, it looks much better. For starters, no XML for me its a big plus.

The libgdx userbase make out the biggest demographic for java gamedev library developers

That's true although I'm not concerned about that. libGDX is largely targeted at Android, I made very explicit that Java 8 is a requirement for dustArtemis (not that you couldn't implement the two or three lambdas present in other ways). That was intentional.

Fun is my primordial concern, and I'm having fun with desktop Java.

Although I do understand that artemis-odb is trying to reach those potential users, having a greater scope, I think its a good idea.

there's very little incentive for people to go out and look for a different implementation in the wild.

Ahh but I will be looking :smile: I find it pretty interesting to look around and see what other things people come up with for dealing with the same problems all frameworks deal with.

[DaanVanYperen]

So you want to play with the tech that interests you and not worry too much about end users. That's valid.

It might just be me, but I sense a certain aversion to maven. ;) You don't need to mavenize (or Gradle) your project just to get it into your own fork of the benchmark. To get your jar ready for benchmarking you just need to install it into your local maven repository.

1. download and set up maven. (needs very little configuration).
2. Maven keeps a local repository. Install your jar into it: mvn install:install-file -Dfile=myjar -DgroupId=net.dust -DartifactId=dustartemis -Dversion=1.0.0 -Dpackaging=jar
3. Fork entity-systems-benchmarks, this will mirror it in your own github repo, then clone it on your local machine.
4. duplicate artemis-odb-0.4.0 folder into dust-artemis-1.0.0.
5. open root/pom.xml and add <module>dust-artemis-1.0.0</module> to the modules tag.
6. open dust-artemis-1.0.0-pom.xml and exchange the net.onedaybeard.artemis:artemis-odb dependency with whatever you named yours in step 2. <dependency><groupId>net.dust</groupId><artifactId>dust-artemis</artifactId><version>1.0.0</version></dependency>
7. import project into eclipse.

That'll get you set up on your end first. I'm certainly interested in Java 8 results for all the frameworks.

This fork of the benchmark has an end user focus, so I suspect the central repository deploy is still a requirement. It's not too trivial to set up if you're unfamiliar with the build automation (be it maven or gradle), but with a bit of effort you should be able to figure it out.

Nb,.You can get your project to deploy on a central repo with Gradle too. Gradle isn't all there yet for some IDE's but since you only target desktop it should be a breeze.

Edit: fixed some formatting issues

[dustContributor]

So you want to play with the tech that interests you and not worry too much about end users.

Exactly.

You don't need to mavenize (or Gradle) your project just to get it into your own fork of the benchmark

Ah but I will anyway. Learning experience for me. I already got the Sonatype thingy approved, now I need to see how to link all this together.

To get your jar ready for benchmarking you just need to install it into your local maven repository...

Thanks for the guide! Though I already got the benchmarks working back in August when the repo was created. I raised an issue like this and junkdog gave me a few pointers. Closed that one because I now plan to upload the integration rather than just run the thing.

I did it the GUI way through Eclipse tho (ie, project -> convert to maven. Then add the dependency through XML editor rather than by hand).

I'm certainly interested in Java 8 results for all the frameworks.

Oh, why didn't you ask before? :stuck_out_tongue:

I got these results yesterday http://pastebin.com/fYTEam0X

Running this version of OpenJDK

openjdk version "1.8.0_40-internal" OpenJDK Runtime Environment (build 1.8.0_40-internal-b04) OpenJDK 64-Bit Server VM (build 25.40-b08, mixed mode

I made sure to let the computer only run the benchmarks, without stuff running in the background. So relative error should be low.

I'm at a loss on why PlainComponentBenchmark bench scales down so fast in dustArtemis. Low insert_remove score for 16k+ entities was kinda expected. I switched to simple ArrayList-like insert/remove with binary search to see how it would perform. Not great apparently.

I didn't benched older versions of artemis-odb nor gdx-artemis because they all run slower than artemis-odb 0.7.0 and all those packed/pooled tests take a lot of time :smiley:

I'm not sure whats the difference between the regular and the "fast" benchmark though. Some particular build setup?

[DaanVanYperen]

fast benchmark uses bytecode weaving. Known bug there though, not sure why it works for you! https://github.com/junkdog/artemis-odb/issues/172

[junkdog]

Ah, no, it's only the weaver module which doesn't compile - not sure if rewriting works, but it's set to use java 6 in the pom (cool if it works though). On Sep 25, 2014 11:05 AM, "Daan van Yperen" notifications@github.com wrote:

fast benchmark uses bytecode weaving. Known bug there though, not sure why it works for you! junkdog/artemis-odb#172 https://github.com/junkdog/artemis-odb/issues/172

— Reply to this email directly or view it on GitHub https://github.com/junkdog/entity-system-benchmarks/issues/8#issuecomment-56792402 .

[DaanVanYperen]

Good to know!

[junkdog]

About _fast, see https://github.com/junkdog/artemis-odb/wiki/Optimizing-Entity-Systems On Sep 25, 2014 11:12 AM, "Adrian Papari" junkdog@angelhill.net wrote:

Ah, no, it's only the weaver module which doesn't compile - not sure if rewriting works, but it's set to use java 6 in the pom (cool if it works though). On Sep 25, 2014 11:05 AM, "Daan van Yperen" notifications@github.com wrote:

fast benchmark uses bytecode weaving. Known bug there though, not sure why it works for you! junkdog/artemis-odb#172 https://github.com/junkdog/artemis-odb/issues/172

— Reply to this email directly or view it on GitHub https://github.com/junkdog/entity-system-benchmarks/issues/8#issuecomment-56792402 .

[DaanVanYperen]

Crap, I thought renaming all my projects _fast would make it run faster. :(

[junkdog]

Nope, you're looking for the _blitz suffix. On Sep 25, 2014 11:14 AM, "Daan van Yperen" notifications@github.com wrote:

Crap, I thought renaming all my projects _fast would make it run faster. :(

— Reply to this email directly or view it on GitHub https://github.com/junkdog/entity-system-benchmarks/issues/8#issuecomment-56793257 .

[dustContributor]

About _fast, see

Ahh, I removed those methods altogether. Only got processEntities. Also removed all the added/removed/changed for single entities. I thought it was kinda silly that the entire Bag wasn't passed to the method.

I thought renaming all my projects _fast would make it run faster.

I remember people suggesting adding "-maxproc" or something like that to the flags of various games to make them run faster :smile:

[dustContributor]

Just a heads up, I'll do this. Last time I revised it, made the pom, got to the GPG signing. I'm just very busy right now :sob:

[junkdog]

Sweet!

[dustContributor]

Well, kinda funny this issue is still up after two years but whatever:

I managed to create a Maven artifact that apparently works fine if you include jitpack.io as one of your repositories, after getting frustrated with sonatype, and bintray after that.

So now I'm trying to implement the benchmarks for dustArtemis. The issues I have is that some of them dont translate well. Like the EntityManglerSystem for example. There is no mechanism to add a whatever-component to an entity, you need a type specific ComponentHandler.

I'm not terribly sure what the 'permutations' array does but I'll see if I figure out a way to adapt it.

EDIT: Retinazer's benchmark seems closer to what I'd need. I'll try with that one as a base.

[dustContributor]

Aaaand here it is #19 Comments?