search

acts-project / acts

Experiment-independent toolkit for (charged) particle track reconstruction in (high energy) physics experiments implemented in modern C++
https://acts.readthedocs.io
Mozilla Public License 2.0
105 stars 168 forks source link

Documentation: hints for debugging #1463

Open timadye opened 2 years ago

timadye commented 2 years ago

It would be nice to have some documentation with hints for debugging ACTS. There are a few specifics:

  1. how to debug C++ with the Python examples
  2. setting breakpoints after shared library loading
  3. debug single-threaded if possible
  4. often simpler to set breakpoints by source file location, rather than using heavily templated symbols.
  5. Debug build can help, but default RelWithDebInfo still possible.

Here is an example I posted to Mattermost that deals with most of these issues. Maybe it would be a good starting point for the documentation. I only have experience of gdb. Andi suggested adding links to the different debuggers.

I didn't yet make a WiP PR, because I don't know where to put this. Should it go on a page of its own, or in a section elsewhere?

It's much easier to run with 1 thread. It's fine to use the Python examples. I edited to set numThreads=1 in acts.examples.Sequencer(). Then ran gdb:

gdb --args python3 Examples/Scripts/Python/full_chain_itk.py
(gdb) b ActsExamples::Sequencer::run()
Function "ActsExamples::Sequencer::run()" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (ActsExamples::Sequencer::run()) pending.

After that, the log includes 8 messages like:

[Detaching after fork from child process 254497]

but those are mostly ROOT running various commands, so not a problem for debugging ACTS. It breaks here:

Breakpoint 1, ActsExamples::Sequencer::run (this=this@entry=0x3df39000) at Examples/Framework/src/Framework/Sequencer.cpp:242
242       Timepoint clockWallStart = Clock::now();

At this point, the program has loaded all the ACTS shared libraries, so you can now set breakpoints directly, not just pending ones. Of course if you want to debug the sequence setup, you'll have to set a breakpoint earlier. But for algorithm execution, Sequencer::run() is a good place to start.

It can be very difficult to set breakpoints using the fully qualified templated symbols, but easy to use the source line, eg.

(gdb) b EigenStepper.ipp:128
Breakpoint 2 at 0x7fffefa2522f: EigenStepper.ipp:128. (10 locations)
(gdb) c

The 10 locations are probably different template instantiations. This one breaks quite quickly, and shows the full (enormous) class name:

Breakpoint 2, Acts::EigenStepper<Acts::StepperExtensionList<Acts::detail::GenericDefaultExtension<double> >, Acts::detail::VoidAuctioneer>::step<Acts::Propagator<Acts::EigenStepper<>, Acts::Navigator>::State<Acts::PropagatorOptions<Acts::ActionList<ActsFatras::detail::SimulationActor<std::mersenne_twister_engine<long unsigned int, 32, 624, 397, 31, 2567483615, 11, 4294967295, 7, 2636928640, 15, 4022730752, 18, 1812433253>, ActsFatras::NoDecay, ActsFatras::InteractionList<ActsFatras::ContinuousProcess<ActsFatras::detail::ScatteringImpl<ActsFatras::detail::Highland>, ActsFatras::ChargedSelector, ActsFatras::EveryParticle, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheBloch, ActsFatras::ChargedSelector, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheHeitler, ActsFatras::AbsPdgSelector<(Acts::PdgParticle)11>, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::Min<ActsFatras::Casts::P> > >, (anonymous namespace)::HitSurfaceSelector> >, Acts::AbortList<ActsFatras::detail::SimulationActor<std::mersenne_twister_engine<long unsigned int, 32, 624, 397, 31, 2567483615, 11, 4294967295, 7, 2636928640, 15, 4022730752, 18, 1812433253>, ActsFatras::NoDecay, ActsFatras::InteractionList<ActsFatras::ContinuousProcess<ActsFatras::detail::ScatteringImpl<ActsFatras::detail::Highland>, ActsFatras::ChargedSelector, ActsFatras::EveryParticle, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheBloch, ActsFatras::ChargedSelector, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheHeitler, ActsFatras::AbsPdgSelector<(Acts::PdgParticle)11>, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::Min<ActsFatras::Casts::P> > >, (anonymous namespace)::HitSurfaceSelector>::ParticleNotAlive, Acts::EndOfWorldReached, Acts::PathLimitReached> > > >(Acts::Propagator<Acts::EigenStepper<Acts::StepperExtensionList<Acts::detail::GenericDefaultExtension<double> >, Acts::detail::VoidAuctioneer>, Acts::Navigator>::State<Acts::PropagatorOptions<Acts::ActionList<ActsFatras::detail::SimulationActor<std::mersenne_twister_engine<unsigned long, 32, 624, 397, 31, 2567483615, 11, 4294967295, 7, 2636928640, 15, 4022730752, 18, 1812433253>, ActsFatras::NoDecay, ActsFatras::InteractionList<ActsFatras::ContinuousProcess<ActsFatras::detail::ScatteringImpl<ActsFatras::detail::Highland>, ActsFatras::ChargedSelector, ActsFatras::EveryParticle, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheBloch, ActsFatras::ChargedSelector, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheHeitler, ActsFatras::AbsPdgSelector<(Acts::PdgParticle)11>, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::Min<ActsFatras::Casts::P> > >, (anonymous namespace)::HitSurfaceSelector> >, Acts::AbortList<ActsFatras::detail::SimulationActor<std::mersenne_twister_engine<unsigned long, 32, 624, 397, 31, 2567483615, 11, 4294967295, 7, 2636928640, 15, 4022730752, 18, 1812433253>, ActsFatras::NoDecay, ActsFatras::InteractionList<ActsFatras::ContinuousProcess<ActsFatras::detail::ScatteringImpl<ActsFatras::detail::Highland>, ActsFatras::ChargedSelector, ActsFatras::EveryParticle, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheBloch, ActsFatras::ChargedSelector, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::EveryParticle>, ActsFatras::ContinuousProcess<ActsFatras::BetheHeitler, ActsFatras::AbsPdgSelector<(Acts::PdgParticle)11>, ActsFatras::Min<ActsFatras::Casts::P>, ActsFatras::Min<ActsFatras::Casts::P> > >, (anonymous namespace)::HitSurfaceSelector>::ParticleNotAlive, Acts::EndOfWorldReached, Acts::PathLimitReached> > > &) const (this=0x7f6ae8, state=...) at Core/include/Acts/Propagator/EigenStepper.ipp:197
197       while (true) {

In this case, it breaks at a different line from that selected, presumably due to inlining the tryRungeKuttaStep lambda. If this is a problem, it might help to compile with -DCMAKE_BUILD_TYPE=Debug, rather than the default RelWithDebInfo.

You can use where to see the full stack trace and all the other gdb commands.

timadye commented 2 years ago

@HadrienG2 followed up on Mattermost, which I copy here:

Glad to see that you sorted this out! Just to clarify one point...

In this case, it breaks at a different line from that selected, presumably due to inlining the tryRungeKuttaStep lambda. If this is a problem, it might help to compile with -DCMAKE_BUILD_TYPE=Debug, rather than the default RelWithDebInfo.

There's actually more than inlining going on in there.

The debugger cannot really break on source lines, since those don't exist anymore in the compiled binary, there are only machine code instructions in there. So what the debugger does, with the help of compiler-generated debug information, is to try and correlate machine instructions with the source line that triggered their generation.

This mapping is ambiguous as a machine instruction can be linked to multiple source instructions (if the compiler merged together redundant computations) and conversely a source line can at times not generate any machine code (if the compiler elided the computation at compile time, or somehow managed to execute the equivalent of multiple source instructions in one machine instruction, then decided that the "main" instruction in that ensemble is not the one that you think).

So what GDB does is to try and break at the point in machine code which is most closely linked to the source line that you were interested in.

stale[bot] commented 2 years ago

This issue/PR has been automatically marked as stale because it has not had recent activity. The stale label will be removed if any interaction occurs.

github-actions[bot] commented 4 months ago

This issue/PR has been automatically marked as stale because it has not had recent activity. The stale label will be removed if any interaction occurs.