search

MAPLE-Lab / auditory-research-suite

Automatically exported from code.google.com/p/auditory-research-suite
0 stars 2 forks source link

Build 173 – Warmup Trial Discrepancies #20

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
As mentioned previously, one of goals of this round of testing was to test out 
the warmup trials. Again, we have document all issues and highlighted them in 
red in the spreadsheet called ‘Build 173 Software Testing Master’, where 
each tab represents each of the 4 statusOrientation settings. In the current 
build (as well as the previous build 169), there were a few cases where the 
number of specified warmup trials did not match the number of warmup trials run 
(i.e. numWarmupTrials=7 and 6 trials were run, etc.). These discrepancies 
seemed to be consistent, regardless of statusOrientation (i.e. consistent 
between tabs), but again no pattern (with regards to specific parameter 
settings) seems to emerge (i.e. there is no clear pattern within the each tab). 
These issues are highlighted in red and documented in columns V and W in the 
spreadsheet. All relevant debug files have been included to assist you with 
these fixes.

I have also included this description as a PDF in case you find it useful to 
have the formatted version with different fonts for technical terms, etc.

Original issue reported on code.google.com by jess.gil...@gmail.com on 5 Feb 2014 at 9:11

Attachments:

GoogleCodeExporter commented 9 years ago

Original comment by zach.br...@datamininglab.com on 12 Feb 2014 at 8:49

GoogleCodeExporter commented 9 years ago 
Michael and Jess, 
We've been able to reproduce and fix these bugs with crashes and incorrect 
number of warmup trials, but we need your input on a special case.

Sometimes in the testing cases you provided, more warmup trials were requested 
than the expected number of experimental trials. In these cases, we have only 
created as many warmup trials as experimental trials. To create more, we would 
have to duplicate some trials, and then the warmup would be longer than the 
actual experiment. 

If you want, we can give as many warmups as requested (though this does not 
make as much sense to us for experimental purposes). 
Or, we could continue capping the number of warmups as we have been doing, and 
provide some debug logging to alert that too many warmups were requested, but 
continue to function normally.

Let us know what functionality you want.

Original comment by zach.br...@datamininglab.com on 12 Feb 2014 at 9:00

GoogleCodeExporter commented 9 years ago 
Fixed in r189.

Found the error in our warmup trial building code. It was causing both the 
crashes and inconsistent number of warmup trials. 

Special case: If more warmup trials than available trials are requested, number 
of warmup trials is capped at the number of available trials.

Original comment by zach.br...@datamininglab.com on 14 Feb 2014 at 9:40