search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
26.99k stars 11.06k forks source link

lldb watchpoint tests are run even if the system has no watchpoints #97834

Open DavidSpickett opened 1 week ago

DavidSpickett commented 1 week ago

This was reported via Discord. During Fedora snapshot builds on PPC, a whole lot of watchpoint tests failed. Including the full list here because some aren't obviously watchpoint related but turn out to be:

Jul 05 06:25:17 ********************
Jul 05 06:25:17 Failed Tests (72):
Jul 05 06:25:17   lldb-api :: commands/watchpoints/hello_watchpoint/TestMyFirstWatchpoint.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/multiple_threads/TestWatchpointMultipleThreads.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/step_over_watchpoint/TestStepOverWatchpoint.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/unaligned-watchpoint/TestUnalignedWatchpoint.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_commands/TestWatchpointCommands.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_commands/command/TestWatchpointCommandLLDB.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_commands/command/TestWatchpointCommandPython.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_commands/condition/TestWatchpointConditionCmd.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_count/TestWatchpointCount.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_disable/TestWatchpointDisable.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_events/TestWatchpointEvents.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_on_vectors/TestValueOfVectorVariable.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_set_command/TestWatchLocationWithWatchSet.py
Jul 05 06:25:17   lldb-api :: commands/watchpoints/watchpoint_size/TestWatchpointSizes.py
Jul 05 06:25:17   lldb-api :: functionalities/breakpoint/scripted_bkpt/TestScriptedResolver.py
Jul 05 06:25:17   lldb-api :: functionalities/completion/TestCompletion.py
Jul 05 06:25:17   lldb-api :: functionalities/inline-stepping/TestInlineStepping.py
Jul 05 06:25:17   lldb-api :: functionalities/location-list-lookup/TestLocationListLookup.py
Jul 05 06:25:17   lldb-api :: functionalities/return-value/TestReturnValue.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/cross_dso/TestCrossDSOTailCalls.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/cross_object/TestCrossObjectTailCalls.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/disambiguate_call_site/TestDisambiguateCallSite.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/disambiguate_paths_to_common_sink/TestDisambiguatePathsToCommonSink.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/disambiguate_tail_call_seq/TestDisambiguateTailCallSeq.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/inlining_and_tail_calls/TestInliningAndTailCalls.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/sbapi_support/TestTailCallFrameSBAPI.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/thread_step_out_message/TestArtificialFrameStepOutMessage.py
Jul 05 06:25:17   lldb-api :: functionalities/tail_call_frames/unambiguous_sequence/TestUnambiguousTailCalls.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentBreakpointsDelayedBreakpointOneWatchpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpointBreakpointSignal.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelaySignalWatch.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayWatchBreak.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayedCrashWithBreakpointWatchpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentManyWatchpoints.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentNWatchNBreak.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalDelayWatch.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalNWatchNBreak.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalWatch.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalWatchBreak.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoBreakpointsOneWatchpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointThreads.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointsOneBreakpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointsOneDelayBreakpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointsOneSignal.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchBreak.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchBreakDelay.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchpointDelayWatchpointOneBreakpoint.py
Jul 05 06:25:17   lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchpointWithDelayWatchpointThreads.py
Jul 05 06:25:17   lldb-api :: functionalities/watchpoint/modify-watchpoints/TestModifyWatchpoint.py
Jul 05 06:25:17   lldb-api :: lang/c/set_values/TestSetValues.py
Jul 05 06:25:17   lldb-api :: lang/c/tls_globals/TestTlsGlobals.py
Jul 05 06:25:17   lldb-api :: lang/cpp/step-through-trampoline/TestStepThroughTrampoline.py
Jul 05 06:25:17   lldb-api :: lang/cpp/trivial_abi/TestTrivialABI.py
Jul 05 06:25:17   lldb-api :: python_api/watchpoint/TestSetWatchpoint.py
Jul 05 06:25:17   lldb-api :: python_api/watchpoint/TestWatchpointIgnoreCount.py
Jul 05 06:25:17   lldb-api :: python_api/watchpoint/TestWatchpointIter.py
Jul 05 06:25:17   lldb-api :: python_api/watchpoint/condition/TestWatchpointConditionAPI.py
Jul 05 06:25:17   lldb-api :: python_api/watchpoint/watchlocation/TestSetWatchlocation.py
Jul 05 06:25:17   lldb-api :: python_api/watchpoint/watchlocation/TestTargetWatchAddress.py
Jul 05 06:25:17   lldb-api :: tools/lldb-dap/databreakpoint/TestDAP_setDataBreakpoints.py
Jul 05 06:25:17   lldb-api :: tools/lldb-server/TestGdbRemoteAttach.py
Jul 05 06:25:17   lldb-api :: tools/lldb-server/TestGdbRemoteProcessInfo.py
Jul 05 06:25:17   lldb-api :: tools/lldb-server/TestGdbRemoteSingleStep.py
Jul 05 06:25:17   lldb-api :: tools/lldb-server/TestGdbRemote_vCont.py
Jul 05 06:25:17   lldb-api :: tools/lldb-server/attach-wait/TestGdbRemoteAttachWait.py
Jul 05 06:25:17   lldb-api :: tools/lldb-server/commandline/TestStubSetSID.py
Jul 05 06:25:17   lldb-shell :: SymbolFile/DWARF/anon_class_w_and_wo_export_symbols.ll
Jul 05 06:25:17   lldb-shell :: SymbolFile/DWARF/clang-ast-from-dwarf-unamed-and-anon-structs.cpp
Jul 05 06:25:17   lldb-shell :: SymbolFile/DWARF/clang-gmodules-type-lookup.c
Jul 05 06:25:17   lldb-shell :: SymbolFile/DWARF/split-optimized.c
Jul 05 06:25:17   lldb-shell :: Watchpoint/ExpressionLanguage.test
Jul 05 06:25:17 
Jul 05 06:25:17 
Jul 05 06:25:17 Testing Time: 426.75s
Jul 05 06:25:17 
Jul 05 06:25:17 Total Discovered Tests: 2990
Jul 05 06:25:17   Excluded         :   26 (0.87%)
Jul 05 06:25:17   Unsupported      :  498 (16.66%)
Jul 05 06:25:17   Passed           : 2374 (79.40%)
Jul 05 06:25:17   Expectedly Failed:   20 (0.67%)
Jul 05 06:25:17   Failed           :   72 (2.41%)

Looking at the logs they all fail with output like:

Jul 05 06:25:06            30: error: Watchpoint creation failed (addr=0x10040038, size=8). 
Jul 05 06:25:06 check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jul 05 06:25:06            31: error: Target supports (0) hardware watchpoint slots. 
Jul 05 06:25:06 check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We should not be running tests on systems where we can tell there are no watchpoints. Or at least we could do a check at the front of each test and print something to tell the user what choices they have here.

I know there is a category for at least the API watchpoint tests, we use it for Windows on Arm for example. That's opt in though.

llvmbot commented 1 week ago

@llvm/issue-subscribers-lldb

Author: David Spickett (DavidSpickett)

This was reported via Discord. During Fedora snapshot builds on PPC, a whole lot of watchpoint tests failed. Including the full list here because some aren't obviously watchpoint related but turn out to be: ``` Jul 05 06:25:17 ******************** Jul 05 06:25:17 Failed Tests (72): Jul 05 06:25:17 lldb-api :: commands/watchpoints/hello_watchpoint/TestMyFirstWatchpoint.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/multiple_threads/TestWatchpointMultipleThreads.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/step_over_watchpoint/TestStepOverWatchpoint.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/unaligned-watchpoint/TestUnalignedWatchpoint.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_commands/TestWatchpointCommands.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_commands/command/TestWatchpointCommandLLDB.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_commands/command/TestWatchpointCommandPython.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_commands/condition/TestWatchpointConditionCmd.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_count/TestWatchpointCount.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_disable/TestWatchpointDisable.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_events/TestWatchpointEvents.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_on_vectors/TestValueOfVectorVariable.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_set_command/TestWatchLocationWithWatchSet.py Jul 05 06:25:17 lldb-api :: commands/watchpoints/watchpoint_size/TestWatchpointSizes.py Jul 05 06:25:17 lldb-api :: functionalities/breakpoint/scripted_bkpt/TestScriptedResolver.py Jul 05 06:25:17 lldb-api :: functionalities/completion/TestCompletion.py Jul 05 06:25:17 lldb-api :: functionalities/inline-stepping/TestInlineStepping.py Jul 05 06:25:17 lldb-api :: functionalities/location-list-lookup/TestLocationListLookup.py Jul 05 06:25:17 lldb-api :: functionalities/return-value/TestReturnValue.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/cross_dso/TestCrossDSOTailCalls.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/cross_object/TestCrossObjectTailCalls.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/disambiguate_call_site/TestDisambiguateCallSite.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/disambiguate_paths_to_common_sink/TestDisambiguatePathsToCommonSink.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/disambiguate_tail_call_seq/TestDisambiguateTailCallSeq.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/inlining_and_tail_calls/TestInliningAndTailCalls.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/sbapi_support/TestTailCallFrameSBAPI.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/thread_step_out_message/TestArtificialFrameStepOutMessage.py Jul 05 06:25:17 lldb-api :: functionalities/tail_call_frames/unambiguous_sequence/TestUnambiguousTailCalls.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentBreakpointsDelayedBreakpointOneWatchpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpointBreakpointSignal.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelaySignalWatch.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayWatchBreak.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentDelayedCrashWithBreakpointWatchpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentManyWatchpoints.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentNWatchNBreak.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalDelayWatch.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalNWatchNBreak.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalWatch.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentSignalWatchBreak.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoBreakpointsOneWatchpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointThreads.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointsOneBreakpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointsOneDelayBreakpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentTwoWatchpointsOneSignal.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchBreak.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchBreakDelay.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchpointDelayWatchpointOneBreakpoint.py Jul 05 06:25:17 lldb-api :: functionalities/thread/concurrent_events/TestConcurrentWatchpointWithDelayWatchpointThreads.py Jul 05 06:25:17 lldb-api :: functionalities/watchpoint/modify-watchpoints/TestModifyWatchpoint.py Jul 05 06:25:17 lldb-api :: lang/c/set_values/TestSetValues.py Jul 05 06:25:17 lldb-api :: lang/c/tls_globals/TestTlsGlobals.py Jul 05 06:25:17 lldb-api :: lang/cpp/step-through-trampoline/TestStepThroughTrampoline.py Jul 05 06:25:17 lldb-api :: lang/cpp/trivial_abi/TestTrivialABI.py Jul 05 06:25:17 lldb-api :: python_api/watchpoint/TestSetWatchpoint.py Jul 05 06:25:17 lldb-api :: python_api/watchpoint/TestWatchpointIgnoreCount.py Jul 05 06:25:17 lldb-api :: python_api/watchpoint/TestWatchpointIter.py Jul 05 06:25:17 lldb-api :: python_api/watchpoint/condition/TestWatchpointConditionAPI.py Jul 05 06:25:17 lldb-api :: python_api/watchpoint/watchlocation/TestSetWatchlocation.py Jul 05 06:25:17 lldb-api :: python_api/watchpoint/watchlocation/TestTargetWatchAddress.py Jul 05 06:25:17 lldb-api :: tools/lldb-dap/databreakpoint/TestDAP_setDataBreakpoints.py Jul 05 06:25:17 lldb-api :: tools/lldb-server/TestGdbRemoteAttach.py Jul 05 06:25:17 lldb-api :: tools/lldb-server/TestGdbRemoteProcessInfo.py Jul 05 06:25:17 lldb-api :: tools/lldb-server/TestGdbRemoteSingleStep.py Jul 05 06:25:17 lldb-api :: tools/lldb-server/TestGdbRemote_vCont.py Jul 05 06:25:17 lldb-api :: tools/lldb-server/attach-wait/TestGdbRemoteAttachWait.py Jul 05 06:25:17 lldb-api :: tools/lldb-server/commandline/TestStubSetSID.py Jul 05 06:25:17 lldb-shell :: SymbolFile/DWARF/anon_class_w_and_wo_export_symbols.ll Jul 05 06:25:17 lldb-shell :: SymbolFile/DWARF/clang-ast-from-dwarf-unamed-and-anon-structs.cpp Jul 05 06:25:17 lldb-shell :: SymbolFile/DWARF/clang-gmodules-type-lookup.c Jul 05 06:25:17 lldb-shell :: SymbolFile/DWARF/split-optimized.c Jul 05 06:25:17 lldb-shell :: Watchpoint/ExpressionLanguage.test Jul 05 06:25:17 Jul 05 06:25:17 Jul 05 06:25:17 Testing Time: 426.75s Jul 05 06:25:17 Jul 05 06:25:17 Total Discovered Tests: 2990 Jul 05 06:25:17 Excluded : 26 (0.87%) Jul 05 06:25:17 Unsupported : 498 (16.66%) Jul 05 06:25:17 Passed : 2374 (79.40%) Jul 05 06:25:17 Expectedly Failed: 20 (0.67%) Jul 05 06:25:17 Failed : 72 (2.41%) ``` Looking at the logs they all fail with output like: ``` Jul 05 06:25:06 30: error: Watchpoint creation failed (addr=0x10040038, size=8). Jul 05 06:25:06 check:14'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Jul 05 06:25:06 31: error: Target supports (0) hardware watchpoint slots. Jul 05 06:25:06 check:14'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``` We should not be running tests on systems where we can tell there are no watchpoints. Or at least we could do a check at the front of each test and print something to tell the user what choices they have here. I know there is a category for at least the API watchpoint tests, we use it for Windows on Arm for example. That's opt in though.