Closed ArekPiekarz closed 4 years ago
@GuillaumeGomez or @antoyo might have an opinion here, as authors of the gtk-test crate :)
Err, you have explored much more options than I did :) .
One thing you could try, if you're on Linux, is to use xvfb
: maybe that will allow you to run the tests in parallel if the window in every instance of xvfb
can have the focus at the same time (I'm really not sure about this, you'll have to check if that is really the case).
Thanks for the xvfb
tip, I like how it doesn't draw windows and doesn't add entries to the taskbar during test execution, although it adds a few seconds due to its initialization. I will experiment with it more.
Assuming I would like to explore running multiple tests on the same GTK library instance, do you have any advice how I could safely destroy all windows at the end of a given test? Or otherwise "deinitialize" the GTK library? I already tried invoking close()
on all top level windows and running event loop until there are no more pending events, but some GtkTooltipWindows still linger after that.
Nevermind, after more research I went with option 3, meaning using the "tests" folder with rusty-fork crate. Updated status of problems:
Xvfb on its own doesn't seem to solve the problems of multithreading if we're using just one instance of it and have tests sharing GTK library, but it's good for hiding the windows.
Thank you for your answers.
Hi, do you know what is the preferred strategy to make integration tests with GTK? The problem with it is there is a global resource that tests may share - the GTK library instance. It manages the lifetime of widgets / GObjects by itself, requires access from only one thread and AFAIK cannot be deinitialized to clean up resources.
I explored the following options while using
gtk-test
crate:1) Put each test in a separate file in the "tests" directory.
Pros: + tests are isolated + automatic registration of tests + quick compile times Cons: - every file is in the same folder, it's hard to distinguish categories of them - slower execution than option 2
\ 2) In the "tests" folder create
mod.rs
with names of subfolders, create subfolders with their ownmod.rs
s, put tests in separate files in subfolders.Pros: + categorization of tests in the form of subfolders + quick compile times + quick test execution + less boilerplate than option 5 Cons: - tests are NOT isolated! they use the same GTK library instance and can access windows/application from other tests - tests cannot in parallel because of GTK limitations - no automatic registration of tests (unless you use
automod
crate, but then IDEs claim the tests are not part of the project and jumping to definitions stops working)\ 3) Using option 2 with
rusty-fork
crate.Pros: + tests are isolated Cons: - execution of tests continues even when one test failed - lots of noise in backtraces -
color-backtrace
crate seems to not work withrusty-fork
backtraces - slower execution than option 2\ 4) Using option 2 with locking a global mutex at the start of every test.
Pros: + better isolation, but not full Cons: - the next test can see still living windows from the previous tests, not being deleted by GTK even after closing them (you can unsafely destroy them manually at the end of tests, but then GTK prints critical errors from its assertions) - developer has to remember to lock the mutex - execution becomes similar in speed to single-threaded
\ 5) Create a separate "integration_tests" folder with subfolders, where each is a crate with it's own
Cargo.toml,
src and tests folders, then put tests in the last ones; add "integration_tests" to workspace in the main project'sCargo.toml
to shareCargo.lock
with the production code.Pros: + tests are isolated + categories are visible in the form of subcrates + automatic registration of tests Cons: - compile times are 2x slower - lots of boilerplate with the additional folders and files - subcrates need to state their dependencies in their
Cargo.toml
s, which leads to duplication and maintenance burden - slower execution than option 2\ As you can see every solution I tried had issues. Do you know if it could be done in a better way? Thanks in advance.