Occasionally failing tests

[msimberg]

This issue is meant to collect the names of failing tests together with links to the failures for investigation. This is a long-running issue and will likely not be closed by one or only a few PRs.

• [ ] condition_variable2
  • https://app.circleci.com/pipelines/github/pika-org/pika/6661/workflows/def23137-e33b-4958-a8c1-d67ac9b63934/jobs/60858

    648/755 Test #640: tests.unit.modules.threading.condition_variable2 ........................................................***Failed   14.95 sec
    /home/circleci/project/pika/libs/pika/threading/tests/unit/condition_variable2.cpp(344): test '!st.stop_requested()' failed in function 'auto test_timed_cv(bool, bool, std::chrono::duration<long, std::ratio<1, 1000> >)::(anonymous class)::operator()(pika::stop_token) const'
    /home/circleci/project/pika/libs/pika/threading/tests/unit/condition_variable2.cpp(344): test '!st.stop_requested()' failed in function 'auto test_timed_cv(bool, bool, std::chrono::duration<long, std::ratio<1, 1000> >)::(anonymous class)::operator()(pika::stop_token) const'
    /home/circleci/project/pika/libs/pika/threading/tests/unit/condition_variable2.cpp(344): test '!st.stop_requested()' failed in function 'auto test_timed_cv(bool, bool, std::chrono::duration<long, std::ratio<1, 1000> >)::(anonymous class)::operator()(pika::stop_token) const'
    Tests failed. 0/0 sanity checks and 3/123 tests failed.

  • disabled in arm64 CI in #741
• [ ] condition_variable3
  • https://cdash.cscs.ch/test/63767909
• [ ] jthread2
  • https://cdash.cscs.ch/test/35872109
• [ ] shared_mutex1:
  • https://cdash.cscs.ch/test/69190832
• [ ] std_thread_scheduler (https://github.com/pika-org/pika/pull/250)
  • https://cdash.cscs.ch/test/12732612
  • https://app.circleci.com/pipelines/github/pika-org/pika/1889/workflows/63918cb6-267a-436a-9eb1-a7718ff70f00/jobs/18999/steps?invite=true#step-103-500
• [ ] stop_token_race
  • https://cdash.cscs.ch/test/34906166
  • https://cdash.cscs.ch/test/111293246
  • https://cdash.cscs.ch/test/112863082
• [ ] thread_pool_scheduler (timeout)
  • https://app.circleci.com/pipelines/github/pika-org/pika/3923/workflows/18bff009-a572-4402-a343-02917e21f379/jobs/37313
• [ ] tss
  • https://app.circleci.com/pipelines/github/pika-org/pika/1245/workflows/08a5ce76-c3dc-4fb2-b557-aa1ce3a96b40/jobs/13394
• [x] ensure_started
  • See https://github.com/pika-org/pika/pull/966#issuecomment-1893406838.
• [x] thread_rescheduling
  • timeout on CCE release build on Jenkins
• [x] split_continuation_clear
  • Apparently only with GCC 14 + stdexec configuration? Probably https://github.com/NVIDIA/stdexec/issues/1426
  • https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/479009878135925/5304355110917878/-/jobs/7723123972#L165
  • https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/479009878135925/5304355110917878/-/jobs/7680430741#L134

[msimberg]

I will attempt to look at the std_thread_scheduler failure for 0.6.0.

[msimberg]

These are some of the more frequently failing tests on CDash: https://cdash.cscs.ch/testOverview.php?project=pika&date=2023-01-31&group=28.

[aurianer]

ARCHIVE (bugs solved):

• [x] any test with APEX enabled:
  • https://cdash.cscs.ch/viewTest.php?onlyfailed&buildid=9673
  • This seems to be due to PAPI. Disabling it removed all typically occuring failures.
• [x] barrier_cpp20
  • https://cdash.cscs.ch/test/32940023
  • fixed in #387
• [x] cublas_matmul (on ault HIP CI)
  • https://cdash.cscs.ch/test/31890299
• [x] executor_parameters_timer_hooks (remove functionality?)
  • https://cdash.cscs.ch/test/11297026
  • Fixed by https://github.com/pika-org/pika/pull/426
• [x] future (deleted in #774, #832)
  • https://github.com/pika-org/pika/runs/8070517198?check_suite_focus=true#step:6:203
• [x] jthread1
  • https://cdash.cscs.ch/test/15194195
  • https://cdash.cscs.ch/test/27109549
  • Probably fixed by #407
• [x] nth_element_range:
  • https://cdash.cscs.ch/test/28784763
  • Fixed by https://github.com/pika-org/pika/pull/412
• [x] partial_sort_range
  • https://cdash.cscs.ch/test/39905086
  • Not explicitly fixed, but algorithms moved to separate repository (#505)
• [x] partition
  • https://cdash.cscs.ch/test/40419799
  • Not explicitly fixed, but algorithms moved to separate repository (#505)
• [x] shared_mutex2:
  • https://cdash.cscs.ch/test/9848329
  • https://cdash.cscs.ch/test/27563480
  • Fixed by https://github.com/pika-org/pika/pull/416
• [x] countif:

  • 17/632 Test  #17: tests.unit.modules.algorithms.algorithms.countif .............................................***Failed    0.32 sec
    using seed: 1650445799
    /Users/runner/work/pika/pika/libs/pika/algorithms/tests/unit/algorithms/countif_tests.hpp(80): test 'num_items == 50u' failed in function 'void test_count_if(ExPolicy &&, IteratorTag) [ExPolicy = const pika::execution::parallel_unsequenced_policy &, IteratorTag = std::forward_iterator_tag]': '23507' != '50'
    0 sanity checks and 1 test failed.

  • using seed: 1660666788
    /dev/shm/pika/src/libs/pika/algorithms/tests/unit/algorithms/countif_tests.hpp(60): test 'num_items == 50u' failed in function 'void test_count_if(IteratorTag) [with IteratorTag = std::random_access_iterator_tag]': '45014' != '50'

  • https://cdash.cscs.ch/test/43664415
  • Not explicitly fixed, but algorithms moved to separate repository (#505)
• [x] unique
  • https://cdash.cscs.ch/test/37089833
  • stacktrace ends in cray_sset_HSW; Cray bug?
  • Not explicitly fixed, but algorithms moved to separate repository (#505)
• [x] split_continuation_clear
  • https://github.com/pika-org/pika/pull/203#issuecomment-1108595339
  • https://app.circleci.com/pipelines/github/pika-org/pika/1778/workflows/1bf8e485-c63d-42d6-b6f0-c0d62607670b/jobs/17990/steps?invite=true#step-103-71
  • https://app.circleci.com/pipelines/github/pika-org/pika/2098/workflows/6411859c-6054-4cf8-8efe-bbc52d437308/jobs/20894/steps
  • https://cdash.cscs.ch/test/31867714
  • maybe fixed by #839