LectureCut should be tested with a few representative videos (fixed GOP, variable GOP) of medium length (~ 20 min maybe).
The tests should be run on different hardware classes (variable core count).
Each step of LectureCut should be analyzed individually. All steps that are currently parallelized using joblib need to be tested with different process numbers. (Is it useful to run with as many processes as cores? Should cores be dynamically allocated for parallel running steps?)
A potential test result could look something like this:
Transcoding Step
2 processes
3 processes
4 processes
$v_1$ (2 cores)
$n_{1,2,1}$ sec
$n_{1,2,2}$ sec
$n_{1,2,3}$ sec
$v_1$ (4 cores)
$n_{1,4,1}$ sec
$n_{1,4,2}$ sec
$n_{1,4,3}$ sec
$v_1$ (8 cores)
$n_{1,8,1}$ sec
$n_{1,8,2}$ sec
$n_{1,8,3}$ sec
$v_2$ (2 cores)
$n_{2,2,1}$ sec
$n_{2,2,2}$ sec
$n_{2,2,3}$ sec
$v_2$ (4 cores)
$n_{2,4,1}$ sec
$n_{2,4,2}$ sec
$n_{2,4,3}$ sec
$v_2$ (8 cores)
$n_{2,8,1}$ sec
$n_{2,8,2}$ sec
$n_{2,8,3}$ sec
$v_1$ is a 20 min video with a fixed GOP of 5 sec/keyframe.
$v_2$ is a 25 min video with a variable GOP between 0.1 sec/keyframe and 10 sec/keyframe.
Testing can be performed in a virtual machine to simulate core counts. Just use the same configuration otherwise to ensure comparability between test results. Each test should ideally be performed multiple times to get a good average.
Implementing Improvements
For each step, the best configuration should automatically be determined based on the core count (if it has a nonnegligible effect) and applied automatically.
~~Major improvements have already been made on the test/time branch!
Please use this branch as the basis for your testing!~~
This has been merged since.
LectureCut should be as performant as possible.
This includes
Analyzing Performance
LectureCut should be tested with a few representative videos (fixed GOP, variable GOP) of medium length (~ 20 min maybe). The tests should be run on different hardware classes (variable core count). Each step of LectureCut should be analyzed individually. All steps that are currently parallelized using
joblib
need to be tested with different process numbers. (Is it useful to run with as many processes as cores? Should cores be dynamically allocated for parallel running steps?) A potential test result could look something like this:Transcoding Step
$v_1$ is a
20 min
video with a fixed GOP of5 sec/keyframe
. $v_2$ is a25 min
video with a variable GOP between0.1 sec/keyframe
and10 sec/keyframe
.Testing can be performed in a virtual machine to simulate core counts. Just use the same configuration otherwise to ensure comparability between test results. Each test should ideally be performed multiple times to get a good average.
Implementing Improvements
For each step, the best configuration should automatically be determined based on the core count (if it has a nonnegligible effect) and applied automatically.