search

chapel-lang / chapel

a Productive Parallel Programming Language
https://chapel-lang.org
Other
1.78k stars 420 forks source link

Unable to live debug multi-locale programs using --gdb #16075

Open sanish81 opened 4 years ago

sanish81 commented 4 years ago

Summary of Problem

Attempting to launch a multi-locale program either with the "--gdb" flag or with CHPL_COMM_USE_GDB set, results in the following error <command-line arg>:1: error: Unexpected flag: "-e"<command-line arg>:1: error: Unexpected flag: "-e"

Compile command:

chpl -g my_app.chpl

Execution command:

./my_app --gdb -nl 2

Configuration Information

$ chpl --version
chpl version 1.22.0
Copyright 2020 Hewlett Packard Enterprise Development LP
Copyright 2004-2019 Cray Inc.
(See LICENSE file for more details)
$ printchplenv --anonymize
CHPL_TARGET_PLATFORM: linux64
CHPL_TARGET_COMPILER: gnu
CHPL_TARGET_ARCH: x86_64
CHPL_TARGET_CPU: unknown
CHPL_LOCALE_MODEL: numa *
CHPL_COMM: gasnet *
  CHPL_COMM_SUBSTRATE: smp *
  CHPL_GASNET_SEGMENT: fast
CHPL_TASKS: qthreads
CHPL_LAUNCHER: smp
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc
CHPL_ATOMICS: intrinsics
  CHPL_NETWORK_ATOMICS: none
CHPL_GMP: gmp
CHPL_HWLOC: hwloc
CHPL_REGEXP: re2
CHPL_LLVM: none
CHPL_AUX_FILESYS: none
gbtitus commented 4 years ago

The smp launcher differs from the others in that it doesn't invoke any system launcher to run the _real executable. Instead all it has to do is to set an environment variable to the desired number of program instances and run the _real directly. Unfortunately the logic common to all launchers which handles invoking a debugger assumes that there is a system launcher wrapping the _real, whose command line can be adjusted to wrap an xterm instead. This is the root cause of the problem. Hopefully there is an elegant way for the common launcher code to recognize that no system launcher is involved, and adjust the command line appropriately.

bradcray commented 4 years ago

@gbtitus: Could the common launcher code check the CHPL_LAUNCHER setting, see that it's the smp case, and special-case it?

e-kayrakli commented 4 years ago

I have been using the UDP conduit for multilocale debugging without any significant issue. It may be a workaround for this issue.

gbtitus commented 4 years ago

Could the common launcher code check the CHPL_LAUNCHER setting ... ?

I think so. We don't pass the CHPL_MAKE_LAUNCHER setting into the launcher library build as a predefined preprocessor variable currently, but we could. Or perhaps we could add the CHPL_LAUNCHER setting to the runtime build's chpl-env-gen.h file and then #include that into the launcher. Checking at execution time would be harder because the actual setting is from the chplenv python scripts rather than the environment; i.e. you could imagine an execution environment in which the chplenv script chose 'smp' as the launcher when CHPL_LAUNCHER wasn't set at all. We'd have to run the python chplenv script out of the launcher to get that right. Checking when we're compiling the launcher library definitely seems easier.

gbtitus commented 4 years ago

I have been using the UDP conduit for multilocale debugging without any significant issue. It may be a workaround for this issue.

Yes, that's what they've switched to now, which somewhat relieves the pressure to get this working.

bradcray commented 4 years ago

Checking at execution time would be harder

Agreed, I think this is something we'd want to decide at compile-time. Either of the techniques you suggest for threading the information into the source seem reasonable to me.