moodlehq / moodle-performance-comparison

Set of shell scripts to run Moodle performance tests using different hardware and configurations and compare results.
GNU General Public License v2.0
76 stars 40 forks source link

Tools to compare Moodle sites and/or branches performance.

Purpose

They can be used to compare:

Features

There are scripts for both the web server and the JMeter server sides.

Requirements

Installation

The installation process differs depending whether you have both the web server and JMeter in the same computer or not.

Web and JMeter servers in the same computer (usually a development computer)

Web server and JMeter running from a different server

Usage

The simplest is to just execute compare.sh, but it will only work in development computers where jmeter is installed in the web server and when you are testing differences between different branches. For other cases the process also differs depending whether you have both web server and JMeter in the same computer or not. Here there is another alternative, you can load your sql dump instead of having a clean brand new site with a fixed dataset, so you can run the generated test plan using real site generated data.

The groupname and description arguments of test_runner.sh are useful to identify the run when comparing results, you can use it to set the branch name, the settings you used or whatever will help you identify which run is it.

Note that you can run the tests as many times as you want, you just need to run after_run_setup.sh and restart_services.sh before running test_runner.sh every time to clean up the site.

It is recommendable that you run all the scripts using the same user (there is no need to use a root user at all) you can use different users to run them (there are no restrictions about it) but be sure that the permissions are correct, it seems to be one of the more common issues when running this tool.

Web and JMeter servers in the same computer, to find performance differences between different branches (usually a development computer)

Web and JMeter servers in the same computer, to find performance differences changing site settings / cache stores

Web server and JMeter running from a different server

Using your own sql dump (Moodle 2.5 onwards)

The installation and configuration is the same, it also depends on if you use the same computer for both web server and JMeter or not, but the usage changes when you want to use your own sql dump, it is not that easy to automate, as you need to specify which course do you want to use as target course and you can not use before_run_setup.sh to generate the test plan and test_files.properties.

testplanfile="/absolute/path/to/testplan.jmx"

datarootbackup="/absolute/path/to/the/dataroot/backup/directory"

testusersfile="/absolute/path/to/testusers.csv"

databasebackup="/absolute/path/to/the/database/backup.sql"

Using your own sql dump (before Moodle 2.5)

Moodle 2.5 introduces the site and the test plan generators, so you can not use them if you are comparing previous branches. But you can:

Advanced usage

Security

This tool in only intended to be used in development/testing environments inside the local network, it would be insecure to expose the project root in a public accessible web server, the same only exposing moodle/ directory:

Troubleshooting