Archive results consistency through averages rather than forcing settings

[dmonllao]

As discussed in the integration chat, would be dangerous to continuously add settings to config.php to have constant and stable results, as:

• Real sites most of the time uses the standard configuration
• Real sites users behaviour is random
• The cost of adapting the settings to new changes would be high
• Would be hard to compare old major releasese with new ones if we add settings to control them

On the other hand tests need more time to finish and they will not be that exact.

We need to change the tool to stress more on loops and test plan repeats and adapt the results collector and/or displayed results to it.

[dmonllao]

As a midpoint I added a compare.sh script using 5 loops and a small (30 users) The results are consistent across multiple execution of the same plan; I will continue adding runs and comparing results because, for what it seems, we would have enough just raising the loops.

[dmonllao]

BTW, the $CFG->session_update_timemodified_frequency hack in config.php has been removed, so there is consistency even with "random" mdl_session updates

[dmonllao]

Once https://tracker.moodle.org/browse/MDL-42198 gets integrated compare.sh will not need to overwrite the loops anymore

[dmonllao]

https://tracker.moodle.org/browse/MDL-42198