Open ctrlcctrlv opened 11 months ago
taviso
commented
11 months ago Hmm, what bash version and distribution is this?
It's supposed to search through just the loaded objects to see if they have any debugging information, and then see if there is a type called unnamed_t.
taviso
commented
11 months ago You seem to get the could not be found message before it goes wrong... looking at the code that cleanup path is super simple - all it does is try to cleanup the compilation units and then resets dwarves.
This seems pretty safe, very tough to guess what went wrong!
My only guess is that you might have some file that confuses dwarves (it's the library we use to parse debug data), but the bug only happens when we're cleaning up? I've never seen this happen - I'll get this fixed if I can reproduce it!
ctrlcctrlv
commented
11 months ago [fred@デブ.狸.agency /tmp]$ bash --version
GNU bash, version 5.2.15(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[fred@デブ.狸.agency /tmp]$ lsb_release -a
LSB Version: n/a
Distributor ID: Arch
Description: Arch Linux
Release: rolling
Codename: n/a
[fred@デブ.狸.agency /tmp]$ neofetch
-` fred@debu.tanuki.agency
.o+` -----------------------
`ooo/ OS: Arch Linux x86_64
`+oooo: Host: FZ40-1 001
`+oooooo: Kernel: 6.5.4-zen2-1-zen
-+oooooo+: Uptime: 1 day, 1 hour, 52 mins
`/:-:++oooo+: Packages: 4523 (pacman), 2 (rpm), 38 (flatpak)
`/++++/+++++++: Shell: bash 5.2.15
`/++++++++++++++: Resolution: 1920x1080
`/+++ooooooooooooo/` DE: Plasma 5.27.8
./ooosssso++osssssso+` WM: KWin
.oossssso-````/ossssss+` WM Theme: Breeze
-osssssso. :ssssssso. Theme: [Plasma], Default [GTK2/3]
:osssssss/ osssso+++. Icons: elementary [Plasma], elementary [GTK2/3]
/ossssssss/ +ssssooo/- Terminal: kitty
`/ossssso+/:- -:/+osssso+- CPU: 11th Gen Intel i7-1185G7 (8) @ 4.800GHz
`+sso+:-` `.-/+oso: GPU: Intel TigerLake-LP GT2 [Iris Xe Graphics]
`++:. `-/+/ Memory: 16994MiB / 31684MiB
.` `/
i'm very curious myself and will try to get you more useful debugging information by trying to reproduce it just in the tty
ctrlcctrlv
commented
11 months ago well this is clearly a concern :laughing:
[fred@デブ.狸.agency ~]$ sudo gdb -p 387606
GNU gdb (GDB) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 387606
[New LWP 387729]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007f188f10359f in __GI___poll (fds=0x7ffc5d13afd0, nfds=1, timeout=249) at ../sysdeps/unix/sysv/linux/poll.c:29
29 return SYSCALL_CANCEL (poll, fds, nfds, timeout);
warning: File "/home/fred/.gdbinit" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
add-auto-load-safe-path /home/fred/.gdbinit
line to your configuration file "/root/.config/gdb/gdbinit".
To completely disable this security protection add
set auto-load safe-path /
line to your configuration file "/root/.config/gdb/gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
info "(gdb)Auto-loading safe path"
(gdb) Quit
(gdb) Quit
(gdb) t
[Current thread is 1 (Thread 0x7f188f260740 (LWP 387606))]
(gdb) bt
#0 0x00007f188f10359f in __GI___poll (fds=0x7ffc5d13afd0, nfds=1, timeout=249) at ../sysdeps/unix/sysv/linux/poll.c:29
#1 0x00007f188e0afbf1 in ?? () from /usr/lib/libcurl.so.4
#2 0x00007f188e0b0072 in ?? () from /usr/lib/libcurl.so.4
#3 0x00007f188e0aa8f6 in curl_multi_wait () from /usr/lib/libcurl.so.4
#4 0x00007f188f2e0707 in ?? () from /usr/lib/libdebuginfod.so.1
#5 0x00007f188ef11b83 in ?? () from /usr/lib/libdw.so.1
#6 0x00007f188ef1ba60 in dwfl_module_getdwarf () from /usr/lib/libdw.so.1
#7 0x00007f188ef7240d in cus__process_dwflmod (dwflmod=0x562b96cac340, userdata=<optimized out>, name=<optimized out>, base=<optimized out>, arg=0x7ffc5d13e430)
at struct/dwarf_loader.c:2420
#8 0x00007f188ef16871 in dwfl_getmodules () from /usr/lib/libdw.so.1
#9 0x00007f188ef6e626 in cus__process_file (filename=0x562b16df9e20 "/usr/lib/libidn2.so.0", fd=3, conf=0x7ffc5d13e5b0, cus=0x562b16dfc1d0) at struct/dwarf_loader.c:2478
#10 dwarf__load_file (cus=0x562b16dfc1d0, conf=0x7ffc5d13e5b0, filename=0x562b16df9e20 "/usr/lib/libidn2.so.0") at struct/dwarf_loader.c:2495
#11 0x00007f188ef6bd4c in cus__load_file (cus=0x562b16dfc1d0, conf=0x7ffc5d13e5b0, filename=0x562b16df9e20 "/usr/lib/libidn2.so.0") at struct/dwarves.c:1825
#12 0x00007f188ef6cb05 in shared_library_callback (info=info@entry=0x7ffc5d13e520, size=size@entry=64, data=data@entry=0x7ffc5d13e5e0) at struct/struct.c:462
#13 0x00007f188f159b78 in __GI___dl_iterate_phdr (callback=callback@entry=0x7f188ef6cae0 <shared_library_callback>, data=data@entry=0x7ffc5d13e5e0) at dl-iteratephdr.c:74
#14 0x00007f188ef6c9fd in generate_standard_struct (list=<optimized out>) at struct/struct.c:644
#15 0x0000562b14d7a825 in execute_builtin (builtin=0x7f188ef6c880 <generate_standard_struct>, words=0x562b16df8a40, flags=0, subshell=0) at execute_cmd.c:4971
#16 0x0000562b14d7b8f2 in execute_builtin_or_function (words=0x562b16df8a40, builtin=0x7f188ef6c880 <generate_standard_struct>, var=0x0, redirects=0x0, fds_to_close=0x562b16d907a0,
flags=0) at execute_cmd.c:5485
#17 0x0000562b14d79fd9 in execute_simple_command (simple_command=0x562b16dfb6a0, pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x562b16d907a0) at execute_cmd.c:4737
#18 0x0000562b14d726ff in execute_command_internal (command=0x562b16dfab30, asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x562b16d907a0) at execute_cmd.c:866
#19 0x0000562b14d71a55 in execute_command (command=0x562b16dfab30) at execute_cmd.c:413
#20 0x0000562b14d5a590 in reader_loop () at eval.c:171
#21 0x0000562b14d57ce6 in main (argc=2, argv=0x7ffc5d13eb28, env=0x7ffc5d13eb40) at shell.c:833
(gdb) frame 7
#7 0x00007f188ef7240d in cus__process_dwflmod (dwflmod=0x562b96cac340, userdata=<optimized out>, name=<optimized out>, base=<optimized out>, arg=0x7ffc5d13e430)
at struct/dwarf_loader.c:2420
warning: Source file is more recent than executable.
2420 Dwarf *dw = dwfl_module_getdwarf(dwflmod, &dwbias);
(gdb) p dwflmod
$1 = (Dwfl_Module *) 0x562b96cac340
(gdb) p *dwflmod
$2 = <incomplete type>
(gdb) frame 7
#7 0x00007f188ef7240d in cus__process_dwflmod (dwflmod=0x562b96cac340, userdata=<optimized out>, name=<optimized out>, base=<optimized out>, arg=0x7ffc5d13e430)
at struct/dwarf_loader.c:2420
2420 Dwarf *dw = dwfl_module_getdwarf(dwflmod, &dwbias);
(gdb) p *dwflmod
$3 = <incomplete type>
(gdb) frame 11
#11 0x00007f188ef6bd4c in cus__load_file (cus=0x562b16dfc1d0, conf=0x7ffc5d13e5b0, filename=0x562b16df9e20 "/usr/lib/libidn2.so.0") at struct/dwarves.c:1825
warning: Source file is more recent than executable.
1825 if (debug_fmt_table[i]->load_file(cus, conf, filename) == 0)
(gdb) p filename
$4 = 0x562b16df9e20 "/usr/lib/libidn2.so.0"
(gdb) p cus
$5 = (struct cus *) 0x562b16dfc1d0
(gdb) p *cus
$6 = {nr_entries = 0, cus = {next = 0x562b16dfc1d8, prev = 0x562b16dfc1d8}}
(gdb) p *cus->next
There is no member named next.
(gdb) p *cus->
cus nr_entries
(gdb) p *cus->cus
Structure has no component named operator*.
(gdb) p cus->cus
$7 = {next = 0x562b16dfc1d8, prev = 0x562b16dfc1d8}
(gdb) p cus->cus.next
$8 = (struct list_head *) 0x562b16dfc1d8
(gdb) p *(cus->cus.next)
$9 = {next = 0x562b16dfc1d8, prev = 0x562b16dfc1d8}
(gdb) p *(cus->cus.next).prev
$10 = {next = 0x562b16dfc1d8, prev = 0x562b16dfc1d8}is this anything?
(gdb) p *nip
$99 = {module = 0x562b16d28830, action_bits = 320}
(gdb) p *nip.module
$100 = {state = 0, functions = {typed = {endaliasent = 0x7f188f23eb20 <main_arena+96>, endetherent = 0x0, endgrent = 0x0, endhostent = 0x0, endnetent = 0x0, endnetgrent = 0x0, endprotoent = 0x0, endpwent = 0x0, endrpcent = 0x0,
endservent = 0x0, endsgent = 0x0, endspent = 0x0, getaliasbyname_r = 0x0, getaliasent_r = 0x0, getcanonname_r = 0x0, getetherent_r = 0x0, getgrent_r = 0x0, getgrgid_r = 0x0, getgrnam_r = 0x0, gethostbyaddr2_r = 0x0,
gethostbyaddr_r = 0x0, gethostbyname2_r = 0x0, gethostbyname3_r = 0x0, gethostbyname4_r = 0x0, gethostbyname_r = 0x0, gethostent_r = 0x0, gethostton_r = 0x0, getnetbyaddr_r = 0x0, getnetbyname_r = 0x0, getnetent_r = 0x0,
getnetgrent_r = 0x0, getntohost_r = 0x0, getprotobyname_r = 0x0, getprotobynumber_r = 0x0, getprotoent_r = 0x0, getpublickey = 0x0, getpwent_r = 0x0, getpwnam_r = 0x0, getpwuid_r = 0x0, getrpcbyname_r = 0x0,
getrpcbynumber_r = 0x0, getrpcent_r = 0x0, getsecretkey = 0x0, getservbyname_r = 0x0, getservbyport_r = 0x0, getservent_r = 0x0, getsgent_r = 0x0, getsgnam_r = 0x0, getspent_r = 0x0, getspnam_r = 0x0, initgroups_dyn = 0x0,
netname2user = 0x0, setaliasent = 0x0, setetherent = 0x0, setgrent = 0x0, sethostent = 0x0, setnetent = 0x0, setnetgrent = 0x0, setprotoent = 0x0, setpwent = 0x0, setrpcent = 0x0, setservent = 0x0, setsgent = 0x0,
setspent = 0x0}, untyped = {0x7f188f23eb20 <main_arena+96>, 0x0 <repeats 63 times>}}, handle = 0x0, next = 0x562b16d28520, name = 0x562b16d28a48 "mymachines"}
(gdb) bt
#0 futex_wait (private=0, expected=2, futex_word=0x7f188f36ea30 <_rtld_global+2608>) at ../sysdeps/nptl/futex-internal.h:146
#1 __GI___lll_lock_wait (futex=futex@entry=0x7f188f36ea30 <_rtld_global+2608>, private=0) at lowlevellock.c:49
#2 0x00007f188f08ff1a in lll_mutex_lock_optimized (mutex=0x7f188f36ea30 <_rtld_global+2608>) at pthread_mutex_lock.c:48
#3 ___pthread_mutex_lock (mutex=0x7f188f36ea30 <_rtld_global+2608>) at pthread_mutex_lock.c:128
#4 0x00007f188f34642e in _dl_add_to_namespace_list (new=0x7f1888000e60, nsid=0) at dl-object.c:33
#5 0x00007f188f342d66 in _dl_map_object_from_fd (name=name@entry=0x7f1888001ee0 "libnss_mymachines.so.2", origname=origname@entry=0x0, fd=-1, fbp=fbp@entry=0x7f188d599ac0, realname=<optimized out>, loader=loader@entry=0x0,
l_type=<optimized out>, mode=<optimized out>, stack_endp=<optimized out>, nsid=<optimized out>) at dl-load.c:1484
#6 0x00007f188f343911 in _dl_map_object (loader=<optimized out>, loader@entry=0x7f188f2ba520, name=name@entry=0x7f1888001ee0 "libnss_mymachines.so.2", type=type@entry=2, trace_mode=trace_mode@entry=0, mode=mode@entry=-1879048190,
nsid=<optimized out>) at dl-load.c:2246
#7 0x00007f188f3472a9 in dl_open_worker_begin (a=a@entry=0x7f188d59a180) at dl-open.c:534
#8 0x00007f188f33c4e1 in __GI__dl_catch_exception (exception=exception@entry=0x7f188d599fe0, operate=operate@entry=0x7f188f347200 <dl_open_worker_begin>, args=args@entry=0x7f188d59a180) at dl-catch.c:237
#9 0x00007f188f346a7a in dl_open_worker (a=a@entry=0x7f188d59a180) at dl-open.c:784
#10 0x00007f188f33c4e1 in __GI__dl_catch_exception (exception=exception@entry=0x7f188d59a160, operate=operate@entry=0x7f188f346a40 <dl_open_worker>, args=args@entry=0x7f188d59a180) at dl-catch.c:237
#11 0x00007f188f346e4c in _dl_open (file=0x7f1888001ee0 "libnss_mymachines.so.2", mode=<optimized out>, caller_dlopen=0x7f188f13d3ef <module_load+175>, nsid=<optimized out>, argc=2, argv=0x7ffc5d13eb28, env=0x562b16e0e2b0)
at dl-open.c:886
#12 0x00007f188f159cd1 in do_dlopen (ptr=ptr@entry=0x7f188d59a3d0) at dl-libc.c:95
#13 0x00007f188f33c4e1 in __GI__dl_catch_exception (exception=exception@entry=0x7f188d59a350, operate=0x7f188f159c90 <do_dlopen>, args=0x7f188d59a3d0) at dl-catch.c:237
#14 0x00007f188f33c603 in _dl_catch_error (objname=0x7f188d59a398, errstring=0x7f188d59a3a0, mallocedp=0x7f188d59a397, operate=<optimized out>, args=<optimized out>) at dl-catch.c:256
#15 0x00007f188f159c41 in dlerror_run (operate=operate@entry=0x7f188f159c90 <do_dlopen>, args=args@entry=0x7f188d59a3d0) at dl-libc.c:45
#16 0x00007f188f159e7f in __libc_dlopen_mode (name=<optimized out>, mode=mode@entry=-2147483646) at dl-libc.c:162
#17 0x00007f188f13d3ef in module_load (module=0x562b16d28830) at nss_module.c:187
#18 0x00007f188f13d94d in __nss_module_load (module=0x562b16d28830) at nss_module.c:302
#19 __nss_module_get_function (module=0x562b16d28830, name=name@entry=0x7f188f1a164d "gethostbyname4_r") at nss_module.c:328
#20 0x00007f188f13b9c1 in __GI___nss_lookup_function (ni=<optimized out>, fct_name=fct_name@entry=0x7f188f1a164d "gethostbyname4_r") at nsswitch.c:137
#21 0x00007f188f0e2fc2 in get_nss_addresses (res=0x7f188d59a7a0, tmpbuf=0x7f188d59a8f0, req=<optimized out>, name=<optimized out>) at ../sysdeps/posix/getaddrinfo.c:631
#22 gaih_inet (tmpbuf=0x7f188d59a8f0, naddrs=<synthetic pointer>, pai=0x7f188d59a770, req=<optimized out>, service=<optimized out>, name=<optimized out>) at ../sysdeps/posix/getaddrinfo.c:1170
#23 __GI_getaddrinfo (name=<optimized out>, service=<optimized out>, hints=<optimized out>, pai=<optimized out>) at ../sysdeps/posix/getaddrinfo.c:2378
#24 0x00007f188e0762c9 in ?? () from /usr/lib/libcurl.so.4
#25 0x00007f188e0793bc in ?? () from /usr/lib/libcurl.so.4
#26 0x00007f188f08c9eb in start_thread (arg=<optimized out>) at pthread_create.c:444
#27 0x00007f188f110dfc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
(gdb) info threads
Id Target Id Frame
1 Thread 0x7f188f260740 (LWP 387606) "bash" 0x00007f188f0894ae in __futex_abstimed_wait_common64 (private=128, cancel=true, abstime=0x0, op=265, expected=387729, futex_word=0x7f188d59b990) at futex-internal.c:57
* 2 Thread 0x7f188d59b6c0 (LWP 387729) "bash" futex_wait (private=0, expected=2, futex_word=0x7f188f36ea30 <_rtld_global+2608>) at ../sysdeps/nptl/futex-internal.h:146
* ```because that's a real hostname…
[fred@デブ.狸.agency ~]$ MYHN=$(hostnamectl --pretty) ; echo $MYHN; getent ahosts `hostnamectl --pretty`
デブ.狸.agency
71.27.48.14 STREAM debu.tanuki.agency
71.27.48.14 DGRAM
71.27.48.14 RAW
[fred@デブ.狸.agency ~]$ dig a `punycode <<< デブ.狸.agency`
; <<>> DiG 9.18.19 <<>> a xn--edk4a.xn--j6x.agency
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32905
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;デブ.狸.agency. IN A
;; ANSWER SECTION:
デブ.狸.agency. 1799 IN CNAME debu.tanuki.agency.
debu.tanuki.agency. 300 IN A 71.27.48.14
;; Query time: 41 msec
;; SERVER: 8.8.4.4#53(8.8.4.4) (UDP)
;; WHEN: Sat Sep 30 18:05:05 EDT 2023
;; MSG SIZE rcvd: 95Seems like an unexpected interaction with debuginfod.
I'll see if I can repro, does it work if you unset DEBUGINFOD_URLS?
bashuses all system RAM then crashes when running structs.shsizeof -a unnamed_tis the culpritSorry, it “searches for the struct”? How? Because strace seems to say that it just loads every .so into bash's memory :joy: