moodlehq / moodle-performance-comparison

Set of shell scripts to run Moodle performance tests using different hardware and configurations and compare results.
GNU General Public License v2.0
75 stars 39 forks source link

Add support for normalized calculations #68

Closed stronk7 closed 4 years ago

stronk7 commented 4 years ago

To avoid some real noise in indicators we are applying a outlier-detection technique (1.5xIQR) and proceeding to normalize them using a pseudo avg that will fit always within the expected values.

Note we don't apply the outlier algorithm recursively neither any other detail. Just 1-pass normalization of noise values.

While 1.5 is a generally accepted value, if we detect that too much is being normalized, we can always go up to 2.3 to just normalize "very rare" outliers.

By default, it's disabled, so everything continues working 100% the same. Only under demand it will become enabled.

You can enable it by adding n=1 or n=true to the url.

At the same time, a bunch of rounding errors have been fixed, display is always with 2 decimal digits but all average calculations are performed with full resolution.

Other small changes are:

sarjona commented 4 years ago

Eloy, thanks for this nice and helpful improvement. I've reviewed the patch and I haven't seen anything wrong there so I'm happy to integrate it! :-)