codership / galera

Synchronous multi-master replication library
GNU General Public License v2.0
447 stars 177 forks source link

Provide command to run Galera tests independently from build process #646

Open ottok opened 11 months ago

ottok commented 11 months ago

What

Currently the build process ends with running a small test suite. Example output:

ake[2]: Entering directory '/builds/mariadb-team/galera-4/debian/output/source_dir/obj-aarch64-linux-gnu'
Running tests...
/usr/bin/ctest --force-new-ctest-process --output-on-failure
Test project /builds/mariadb-team/galera-4/debian/output/source_dir/obj-aarch64-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................   Passed    3.03 sec
    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................   Passed    2.86 sec
    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed    1.58 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................   Passed    0.06 sec
    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed    9.15 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................   Passed    5.38 sec
    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.07 sec
100% tests passed, 0 tests failed out of 7
Total Test time (real) =  22.15 sec

Could you provide a command to run these tests independently from the build process?

Running the tests need the built binaries. Running tests may require access to source code - but not to build dependencies (running the tests can't require rebuilding everything from scratch).

Why

Debian has an extensive CI system. Currently the system runs very superficial tests and thus the Debian CI system shows 'neutral' for Galera:

image

If there was a command to run 'real' tests, I could integrate it into Debian CI and have real confidence that if the tests ran using binaries built for Debian version X on architecture Y, then those Galera binaries actually as usable.

ottok commented 10 months ago

@temeo Could you provide a command to run these tests independently from the build process? Or is it hugely complex to do? Can you advice what could perhaps be feasible?

ottok commented 5 months ago

This is tracked in Debian as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053333 and help is wanted. I assume also other distros would benefit if there was some basic smoke test type of thing that can run to verify that the compiled binaries are functioning.