Wmes as list

[msche]

Refactored code so that WMEs are using java lists instead of custom linked list.

[marinier]

Unfortunately, jsoar did not use standard lists for performance reasons. The original analysis is described here:

https://github.com/soartech/jsoar/wiki/ListPerformanceAnalysis

ListHead (aka intrusive) lists are much faster for iteration, and jsoar does a LOT of list iteration. Additionally, jsoar's use of intrusive lists closely mirrors csoar's implementation, which helps port changes from csoar to jsoar, as the code is very similar in most cases.

[marinier]

Sorry, I don't mean to be discouraging, as this looks like it was a lot of work. Given the original analysis was done 8 years ago, it may be that Java has improved in ways that reduce or eliminate the performance difference. If that were the case, I agree that using standard lists would probably be easier for future development (it remains to be seen on the csoar porting, but that doesn't happen regularly anyway).

If an analysis could show that the changes were as performant as the current implementation, I'd consider this more deeply.

[msche]

Hi Marinier, no worries what I'm trying to do (for myself) is making the core a bit more understandable and start to understand it. Best way for me is doing some refactoring. How could I test the performance of JSoar to see whether my changes made it better or worse?

[marinier]

There is a jsoar-performance-testing subproject you could try. It was originally designed to compare jsoar and csoar, but they have diverged enough (which is to say, jsoar has not updated with the latest csoar changes in some years) that this doesn't work well for agents that use any of the advanced features (epmem, smem, chunking, RL, SVS). But for agents that are just straight rules (e.g., the simplest waterjugs, TOH, etc.) it's still works.

Anyway, what you really want to compare is two versions of jsoar. You can use jsoar-performance-testing for that, too. For this purpose I would suggest you look at jsoar-performance-testing/src/main/resources/example_configuration.yaml you can see how to make a test configuration (I recommend doing everything in the default settings, unless you really want to vary settings per test):

• You can have pre-built jsoar-core jars for each version you want to test and configure the test to use them all.
• You can disable csoar (or throw that in the mix, too, if you want).
• For an agent, I recommend using a basic TOH or waterjug agent with the goal rule removed (or at least the halt removed). This will let the agents run for as long as we want (normally they would not run long enough to make a good benchmark).
  • I would start with the versions that just execute in the top-state, although you could also add the substate versions. I would disable learning, at least to start, although you could have a version for that, too (it won't compare with csoar, since the learning mechanisms have changed and the same agent may not even work in both jsoar and csoar with learning).
  • Note there are already single-file versions of these agents in /jsoar-core/src/test/resources/org/jsoar/kernel, e.g., /jsoar-core/src/test/resources/org/jsoar/kernel/FunctionalTests_testTowersOfHanoi.soar
  • If you use waterjug, be sure to set a random seed in the config so the agents are running the exact same decisions.
• If you removed the halting, you should set a decision cycle count. Maybe 100000 would be good (long enough to not be too noisy, not so long that the test takes forever).
• You could set the run count to something like 10 (i.e., repeat the test 10 times for each configuration)
• For the most fair comparison, a warmup count of maybe 5 would be good (so the java hotspotting is likely to have already taken place).

That's pretty much it. There are other settings you can play with -- it's easy to get lost in trying lots of different combinations of things :) The output will be a csv file with basic stats (e.g., average, min, max run times). You can easily import it into something like Excel.