Instance_create fails in Fortran if debugger is attached

[AreWeDreaming]

I am trying to debug a the muscle3 wrapper of a Fortran code, but for some reason I get a crash at LIBMUSCLE_Instance_create ONLY when I attach gdbserver. This is the stacktrace:

libstdc++.so.6!__cxxabiv1::__cxa_throw(void * obj, std::type_info * tinfo, void (*)(void *) dest) (\home\conda\feedstock_root\build_artifacts\gcc_compilers_1706816862910\work\libstdc++-v3\libsupc++\eh_throw.cc:80)
libmuscle.so!libmuscle::impl::SettingsManager::get_setting(ymmsl::impl::Reference const&, ymmsl::impl::Reference const&) const [clone .cold] (Unknown Source:0)
libmuscle.so!libmuscle::impl::Instance::Impl::set_local_log_level_() (Unknown Source:0)
libmuscle.so!libmuscle::impl::Instance::Impl::Impl(int, char const* const*, std::unordered_map<ymmsl::impl::Operator, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::hash<ymmsl::impl::Operator>, std::equal_to<ymmsl::impl::Operator>, std::allocator<std::pair<ymmsl::impl::Operator const, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > > const&, libmuscle::impl::InstanceFlags) (Unknown Source:0)
libmuscle.so!libmuscle::impl::Instance::Instance(int, char const* const*, std::unordered_map<ymmsl::impl::Operator, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::hash<ymmsl::impl::Operator>, std::equal_to<ymmsl::impl::Operator>, std::allocator<std::pair<ymmsl::impl::Operator const, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > > const&, libmuscle::impl::InstanceFlags) (Unknown Source:0)
libmuscle.so!LIBMUSCLE_Instance_create_ (Unknown Source:0)
libmuscle_fortran.so!__libmuscle_MOD_libmuscle_instance_create (Unknown Source:0)

I am using:

• gcc: gcc (conda-forge gcc 11.4.0-5) 11.4.0
• gdb: GNU gdb (GDB) 14.2
• muscle3: 0.7.1

If it helps I can recompile muscle3 with debug symbols to shed more light on this. I am stumped.

I made sure that my configuration.ymmsl and corresponding debug_configuration.ymmsl are identical except that the latter has:

  micro:
    executable: gdbserver
    args:
    -  "--no-startup-with-shell"
    - ":5679"
    - "a.out"

instead of:

  micro:
    executable: a.out

configuration.ymmsl works just fine (except for the bugs in a.out that I am trying to fix), but when launched with gdbserver I reliable get the crash reported above.

[LourensVeen]

Hmm, that's strange. I think MUSCLE3 builds a debug version of the library, but I don't remember whether it only gets used to build the tests, or if it's also installed by default. You could see if there's a lib/libmuscle_d.so installed and link to that, that should give a bit more information as to where it crashes exactly.

Is there anything in the manager log about the instance having registered? That should have happened by the time it gets here, and if it hasn't then we won't have settings and that may cause an issue maybe. But I don't see how that could happen...

[AreWeDreaming]

muscle_manager 2024-03-27 07:49:33,381 INFO    libmuscle.manager.instance_manager: Planned macro on Resources(omega26.gat.com: 0)
muscle_manager 2024-03-27 07:49:33,381 INFO    libmuscle.manager.instance_manager: Planned micro on Resources(omega26.gat.com: 0)
muscle_manager 2024-03-27 07:49:33,382 INFO    libmuscle.manager.instance_manager: Instantiating macro on Resources(omega26.gat.com: 0)
muscle_manager 2024-03-27 07:49:33,382 INFO    libmuscle.manager.instance_manager: Instantiating micro on Resources(omega26.gat.com: 0)
muscle_manager 2024-03-27 07:49:46,627 INFO    libmuscle.manager.mmp_server: Registered instance macro 
muscle_manager 2024-03-27 07:49:54,799 INFO    libmuscle.manager.mmp_server: Registered instance micro
macro 2024-03-27 07:49:58,313 ERROR   macro: Failed to connect

[AreWeDreaming]

I also cannot find lib/libmuscle_d.so only a libymmsl_d.so. The makefile didn't give me any obvious ways to build muscle3 with debug symbols.

[maarten-ic]

Looking at the backtrace, set_local_log_level_ seems to expect that exceptions are thrown: see https://github.com/multiscale/muscle3/blob/develop/libmuscle/cpp/src/libmuscle/instance.cpp#L1140

Could it be that your gdbserver is set up to break on all thrown exceptions, even when that exception would normally be caught?

[AreWeDreaming]

Oh dear, I feel very silly now. I can indeed get past this exception. I never tried because I did not expect it to break on handled exceptions but this is indeed what it is doing. Thanks for the help!

[LourensVeen]

Ah, and now I'm feeling silly too, because I've seen this before, so I feel like I should have recognised it. Well, glad the confusion is resolved. I'll make a separate issue for that debug build.