Implement VM arguments support in new-caliper

[GoogleCodeExporter]

Apologies that I'm mixing behavior specification and implementation suggestions 
together in what follows; it's just easier that way for now.

We add a VmArguments class which encapsulates a Map<String,
String> (logically, a Map<LogicalParamName, ActualArgumentsSpaceSeparated>, or 
ListMultimap instead of space-separation if preferred).

e.g. it might map the logical param name "memory" to a string like
"-Xmx2g -Xss1m", or whatever. VM parameters should *always* be associated with 
a logical name, like "memory" in this example, because that is what allows them 
to be overridden.

First this map comes from the VM defaults in the config file.  Then the
instrument can "override" its own choices -- wherever it used the same key, its 
values replace the others.  Next the benchmark class, and finally the caliper 
command line. In each case, if the key already exists the new options replace 
the previous ones, otherwise, they add on. (Yeah, I just explained what Map.put 
does.)

So some keys will become sort of "standardized" because they're the names
we use by default in the global configs, but there's nothing specifically
magical about those names otherwise.

When it's time to actually execute the VM, we just have to basically
concatenate all the values() of that map.

As it stands today, the VM defaults and instrument defaults probably don't
have a "LogicalParamName" in there yet. They need to.

The result is that in effect every benchmark will wind up with lots and lots of 
axes, but that's okay, because unless the user went out of their way these are 
all going to be *singleton* axes, and the way we handle those in the UI will be 
unobtrusive.

Original issue reported on code.google.com by kevinb@google.com on 8 May 2012 at 9:28

[GoogleCodeExporter]

My vote is for the VmArguments class (or maybe VmSpec?) to have a specific 
field for every property that we have identified as being significant in 
distinguishing JVMs.  Two reasons:

1) It will make sure that we do the right thing for -Xmx2g vs -Xmx2048m
2) It will force us into making sure that every property that we identify as 
being "important" in differentiating VMs is present and being compared 
correctly.

Original comment by gak@google.com on 8 May 2012 at 9:33

[GoogleCodeExporter]

I believe the design described above makes what you describe unnecessary.

The defaults in .caliper/config will include something like "memory=-Xmx2500m". 
If the user wants to override that, they have to use the same key ("memory"):  
"memory=-Xmx5g".  If they do this in a @VmParam, "memory" has to be the field 
name; if at the command line it's -Jmemory=, etc.

If this seems screwy, at least give it the benefit of the doubt until we can 
discuss it more fully. How to solve this was one of our most vexing problems 
for years, and I believe Jesse and I sunk *dozens* of hours into trying to find 
the simplest solution possible, and this is where we ended up.

Original comment by kevinb@google.com on 8 May 2012 at 9:58

[GoogleCodeExporter]

Greg has a completely different approach for this problem that we *think* will 
work and I think is going to be brilliant if it does.

Basically, we stop worrying about overload resolution of vm args, just relying 
on the fact that (apparently) "last wins" in the JDK we tested. Then for 
reporting results, we ignore all those text flags and actually *detect* the 
settings that were in effect for the VM -- which is a way better approach for 
all kinds of reasons.

This should dispense with the need to give groups of options a user-chosen 
logical name, since whatever VM facility Greg found to dump out the settings 
already has a decent name for everything!

Original comment by kevinb@google.com on 23 May 2012 at 10:43

[GoogleCodeExporter]

We have some pretty great support for VM arguments now.  I'm quite happy with 
it.

Original comment by gak@google.com on 1 Nov 2012 at 9:06

• Changed state: Fixed