seqan / seqan3

The modern C++ library for sequence analysis. Contains version 3 of the library and API docs.
https://www.seqan.de
Other
412 stars 82 forks source link

[TEST] Skip parallel tests on machines having a single CPU #3303

Closed sanvila closed 2 weeks ago

sanvila commented 2 weeks ago

Otherwise this is what happens:

The following tests FAILED:
        5762 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.spmc_sum (Failed)
        5763 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.mpsc_sum (Failed)
        5764 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.mpmc_sum (Failed)
        5767 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.spmc_dynamicsize (Failed)
        5768 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.spmc_fixedsize (Failed)
        5769 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.mpsc_dynamicsize (Failed)
        5770 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.mpsc_fixedsize (Failed)
        5771 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.mpmc_dynamicsize (Failed)
        5772 - contrib/parallel/buffer_queue_parallel_test::buffer_queue.mpmc_fixedsize (Failed)
Errors while running CTest
seqan-actions commented 2 weeks ago

Documentation preview available at https://docs.seqan.de/preview/seqan/seqan3/3303

codecov[bot] commented 2 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.13%. Comparing base (3b4f644) to head (54e011d). Report is 2 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #3303 +/- ## ======================================= Coverage 98.13% 98.13% ======================================= Files 271 271 Lines 11948 11948 Branches 104 104 ======================================= Hits 11725 11725 Misses 223 223 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

eseiler commented 2 weeks ago

Hey @sanvilla,

thanks for the PR!

As this bug results from dividing the thread count by two in https://github.com/seqan/seqan3/blob/3b4f6442f632b0b64f0a7d3172f2d71307b6af1d/test/unit/contrib/parallel/buffer_queue_parallel_test.cpp#L24

I would suggest adapting the test instead

diff --git a/test/unit/contrib/parallel/buffer_queue_parallel_test.cpp b/test/unit/contrib/parallel/buffer_queue_parallel_test.cpp
index 6bf5b553e..34c8def1a 100644
--- a/test/unit/contrib/parallel/buffer_queue_parallel_test.cpp
+++ b/test/unit/contrib/parallel/buffer_queue_parallel_test.cpp
@@ -15,11 +15,8 @@
 template <typename sequential_push_t, typename sequential_pop_t>
 void test_buffer_queue_wait_status()
 {
-    size_t thread_count = std::thread::hardware_concurrency();
-
-    // limit thread count as virtualbox (used by Travis) seems to have problems with thread congestion
-    if (thread_count > 4)
-        thread_count = 4;
+    // At least two threads (one producer and one consumer), at most 4 threads (avoid congestion).
+    size_t thread_count = std::clamp<size_t>(std::thread::hardware_concurrency(), 2u, 4u);

     size_t writer_count = thread_count / 2;
     if constexpr (sequential_push_t::value)
@@ -129,11 +126,9 @@ void test_buffer_queue_wait_throw(size_t initialCapacity)
     }

     volatile std::atomic<size_t> chk_sum2 = 0;
-    size_t thread_count = std::thread::hardware_concurrency();

-    // limit thread count as virtualbox (used by Travis) seems to have problems with thread congestion
-    if (thread_count > 4)
-        thread_count = 4;
+    // At least two threads (one producer and one consumer), at most 4 threads (avoid congestion).
+    size_t thread_count = std::clamp<size_t>(std::thread::hardware_concurrency(), 2u, 4u);

     size_t writer_count = thread_count / 2;
     if constexpr (sequential_push_t::value)

Do you want to adapt your PR or should we push the changes?

SGSSGene commented 2 weeks ago

@sanvila : What you have found, is definitely a bug in our tests. But, I am curious, under which circumstances do you actually have only one core?

sanvila commented 2 weeks ago

Do you want to adapt your PR or should we push the changes?

Feel free to reject the PR and fix the issue as you see fit, as long as it works on single-CPU systems.

Thanks a lot!

sanvila commented 2 weeks ago

What you have found, is definitely a bug in our tests. But, I am curious, under which circumstances do you actually have only one core?

Good question! I'm doing QA work for Debian, using virtual machines in the cloud, to ensure that all packages may be built from source.

The official Debian autobuilders are physical machines with four or more CPUs, and as a result, that’s the most tested configuration. So, to compensate, I’m using systems with one and two CPUs for my QA work. After all, Debian aims to be the universal operating system, suitable for both desktops and servers.