search

van-smith / OPBM

Open Productivity Benchmark
1 stars 0 forks source link

individual measurement timestamps are not recorded in xml #113

Open jimmacd opened 12 years ago

jimmacd commented 12 years ago

in analyzing our test results I find that the individual measurements are not time stamped. The time stamp field is the same for all measurement records which I assume is the time of the start of test. A time stamp should be recorded for each measurement. I'm simply looking for a way to track the execution of the benchmark in time. In other benchmarks I sort by time stamp of the measurement. I'm open to suggestions if I've wandered outside a design meme here.

van-smith commented 12 years ago

Hi Jim, Your expectations are appropriate and this should be the behavior for all measurements.

ghost commented 12 years ago

Manifest.xml records timestamps for every output item intercepted through the HUD. While those timing fields are not carried forward directly to the results presently, a lookup can be performed to find their timing source lines via the uuid fields.

I've sent you an email which outlines these steps. Here is a summary (jim_explanation.txt) which explains the process (for jim_example.xml):

Take the timing on line 28. It has the uuid field "ba042a63-3dac-43e7-9a32-d88abf1d47dd". This value can be searched to find the byAtom timing lines further down in manifest.xml.

Line 28's parent Xml tag contains the attribute manifestworkletuuid field "25375277-1cfe-46b1-8b50-0217b95a729a". This uuid value can be searched to find the original worklet within the benchmark that initiated the test/score, such as line 10. It can also be used to find the original details of the run, which begin on line 34.

Once you have identified those two sources, the full range of data tracking is exposed. Looking for the single "success" tag (or possible multiple "failure" tags), the relative xml tag "abstract.outputs" can be found, which begins on line 43. Iterating through each of those output entries, and testing for text prefixes of "timing," and the same description as the line 28 name above ("Close Microsoft Publisher 2010"), the exact timestamp can be accessed.

The text "(colon)" should be changed to ":" for processing. For whatever reason, Java's DOM doesn't like generic : fields in attribute values.

ghost commented 12 years ago

The millisecond field is now being copied through into results.xml. I can also be added to the csv files if required. If so, please indicate where you'd like it added. Additionally, I can copy the date+time field as well, were a specific fixed quantity required, rather than just a sortable reference for the list of entries.

van-smith commented 12 years ago

I verified that the system clock time in milliseconds is now being reported.