search

moodlehq / moodle-performance-comparison

Set of shell scripts to run Moodle performance tests using different hardware and configurations and compare results.
GNU General Public License v2.0
75 stars 39 forks source link

Add a base branch #5

Closed dmonllao closed 11 years ago

dmonllao commented 11 years ago

I would paste the conversation with @stronk7 but is in Spanish so is better to resume it.

Thinking about our CI jobs and historic data about them, we don't have enough with $beforebranch and $afterbranch, if there is a tool_generator change between $beforebranch and $afterbranch the results can not be compared, we need to define a common base branch and keep the generators as static as possible.

$basebranch should also be included to the information passed to the test run.

dmonllao commented 11 years ago

8c966275a87dbf716166cff01043793fc9570c4a

dmonllao commented 11 years ago

Added a base branch pre-process to checkout the base point; the backup is done after this point and before the $beforebranch checkout, so both before and after branches are upgraded from this point.

More config settings added, each branch/hash can specify a different repository so there is no restriction to single repository anymore, most of the time it will probably be the same one though.

About those bases, we should provide/maintain a list of those breakpoints or at least have a hardcoded list of options for $basebranch in https://github.com/moodlehq/moodle-performance-comparison/blob/master/webserver_config.properties.dist so users can know which git history points can be used without having problems because of tool_generator code changes, I would go for the second one so we just have one reference point to change; could be also good to add something on the jira side (like a new label) to tag tool_generator issues that affects the performance results and update webserver_config.properties.dist with the new breakpoint according to it, any better alternative?

dmonllao commented 11 years ago

Reopening after talking with @stronk7, we will branch the project according to moodle/moodle major branches and we will hardcode the last commit which contains tool_generator changes in webserver_config.properties.dist, so users don't need to worry about setting it.

We should avoid changes in stable as much as possible so otherwise we will need to change the last valid commit.

Also returning to single repository.

dmonllao commented 11 years ago

76b988d8ddf5bb96886905acd689838e2d630e5c