search

igankevich / mesonic

Mesonic: A Vim plugin for Meson build system
46 stars 7 forks source link

Add MesonTest command that wraps `meson test` #19

Closed igankevich closed 2 years ago

igankevich commented 2 years ago

See the discussion in #16.

albfan commented 2 years ago

I can provide feedback with a project I maintain: https://gitlab.gnome.org/GNOME/gitg

$ git clone https://gitlab.gnome.org/GNOME/gitg
$ meson build
$ meson test -C build --help
usage: meson test [-h] [--repeat REPEAT] [--no-rebuild] [--gdb] [--gdb-path GDB_PATH] [--list] [--wrapper WRAPPER] [-C WD] [--suite SUITE] [--no-suite SUITE] [--no-stdsplit] [--print-errorlogs] [--benchmark] [--logbase LOGBASE]
                  [--num-processes NUM_PROCESSES] [-v] [-q] [-t TIMEOUT_MULTIPLIER] [--setup SETUP] [--test-args TEST_ARGS]
                  [args ...]

positional arguments:
  args                                                            Optional list of test names to run. "testname" to run all tests with that name, "subprojname:testname" to specifically run "testname" from "subprojname", "subprojname:" to
                                                                  run all tests defined by "subprojname".

options:
  -h, --help                                                      show this help message and exit
  --repeat REPEAT                                                 Number of times to run the tests.
  --no-rebuild                                                    Do not rebuild before running tests.
  --gdb                                                           Run test under gdb.
  --gdb-path GDB_PATH                                             Path to the gdb binary (default: gdb).
  --list                                                          List available tests.
  --wrapper WRAPPER                                               wrapper to run tests with (e.g. Valgrind)
  -C WD                                                           directory to cd into before running
  --suite SUITE                                                   Only run tests belonging to the given suite.
  --no-suite SUITE                                                Do not run tests belonging to the given suite.
  --no-stdsplit                                                   Do not split stderr and stdout in test logs.
  --print-errorlogs                                               Whether to print failing tests' logs.
  --benchmark                                                     Run benchmarks instead of tests.
  --logbase LOGBASE                                               Base name for log file.
  --num-processes NUM_PROCESSES                                   How many parallel processes to use.
  -v, --verbose                                                   Do not redirect stdout and stderr
  -q, --quiet                                                     Produce less output to the terminal.
  -t TIMEOUT_MULTIPLIER, --timeout-multiplier TIMEOUT_MULTIPLIER  Define a multiplier for test timeout, for example when running tests in particular conditions they might take more time to execute. (<= 0 to disable timeout)
  --setup SETUP                                                   Which test setup to use.
  --test-args TEST_ARGS                                           Arguments to pass to the specified test(s) or all tests

Probably support --test-args is interesting, but a test suite that depends on external arguments looks like integration, not unit test. --suite, --benchmark too

For autocomplete:

$ meson test -C build --list
tests-libgitg
tests-gitg

Plain execution:

$ meson test -C build
ninja: Entering directory `/home/alberto/projects/vala/gitg/build'
ninja: no work to do.
1/2 tests-libgitg        OK              0.09s
2/2 tests-gitg           OK              0.22s

Ok:                 2   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to /home/alberto/projects/vala/gitg/build/meson-logs/testlog.txt

To run individual test:

$ sudo meson test -C build tests-gitg
ninja: Entering directory `/home/alberto/projects/vala/gitg/build'
ninja: no work to do.
1/1 tests-gitg        OK              0.22s

Ok:                 1   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to /home/alberto/projects/vala/gitg/build/meson-logs/testlog.txt

Test can run in verbose mode:

$ meson test -C build tests-gitg -v
ninja: Entering directory `/home/alberto/projects/vala/gitg/build'
ninja: no work to do.
1/1 tests-gitg        RUNNING       
>>> MALLOC_PERTURB_=250 /home/alberto/projects/vala/gitg/build/tests/gitg/tests-gitg

1/1 tests-gitg        OK              0.23s
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
stdout:
# random seed: R02S4aeda1817844fe8aed51f258a6e9d6c6
1..21
# Start of GitgTestCheckoutRef tests
ok 1 /GitgTestCheckoutRef/checkout-branch
ok 2 /GitgTestCheckoutRef/checkout-branch-safe
ok 3 /GitgTestCheckoutRef/checkout-branch-conflict
# End of GitgTestCheckoutRef tests
# Start of GitgTestMergeRef tests
ok 4 /GitgTestMergeRef/merge-simple
ok 5 /GitgTestMergeRef/merge-not-head
ok 6 /GitgTestMergeRef/merge-not-head-would-have-conflicted
ok 7 /GitgTestMergeRef/merge-theirs-conflicts-no-checkout
ok 8 /GitgTestMergeRef/merge-theirs-conflicts-checkout
ok 9 /GitgTestMergeRef/merge-theirs-dirty-stash
ok 10 /GitgTestMergeRef/merge-theirs-not-master-conflicts-checkout
ok 11 /GitgTestMergeRef/merge-theirs-not-master-conflicts-checkout-dirty
ok 12 /GitgTestMergeRef/merge-theirs-not-master-conflicts-checkout-dirty-no-stash
# End of GitgTestMergeRef tests
# Start of GitgTestCherryPickCommit tests
ok 13 /GitgTestCherryPickCommit/merge-theirs-conflicts-checkout
ok 14 /GitgTestCherryPickCommit/cherry-pick-simple
ok 15 /GitgTestCherryPickCommit/cherry-pick-not-head
ok 16 /GitgTestCherryPickCommit/cherry-pick-not-head-would-have-conflicted
ok 17 /GitgTestCherryPickCommit/cherry-pick-theirs-conflicts-no-checkout
ok 18 /GitgTestCherryPickCommit/cherry-pick-theirs-dirty-stash
ok 19 /GitgTestCherryPickCommit/cherry-pick-theirs-not-master-conflicts-checkout
ok 20 /GitgTestCherryPickCommit/cherry-pick-theirs-not-master-conflicts-checkout-dirty
ok 21 /GitgTestCherryPickCommit/cherry-pick-theirs-not-master-conflicts-checkout-dirty-no-stash
# End of GitgTestCherryPickCommit tests
stderr:

(process:206847): Gtk-CRITICAL **: 00:41:03.508: gtk_style_context_add_provider_for_screen: assertion 'GDK_IS_SCREEN (screen)' failed
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

Ok:                 1   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to /home/alberto/projects/vala/gitg/build/meson-logs/testlog.txt

I think that covers mostly all

igankevich commented 2 years ago

I added MesonTest and MesonBenchmark commands. Please, check.

albfan commented 2 years ago

Looks perfect to me. Just missing friendly warn to user if you didn't :MesonInit

igankevich commented 2 years ago

Just missing friendly warn to user if you didn't :MesonInit

May be we should run it automatically with the default directory name, e.g. "build"? We already do this for the syntastic plugin. Of course, the name can be overridden with some variable.

albfan commented 2 years ago

I always prefer warnings that automagic, but these are just details.

For me this can be closed