search

apache / trafficserver

Apache Traffic Server™ is a fast, scalable and extensible HTTP/1.1 and HTTP/2 compliant caching proxy server.
https://trafficserver.apache.org/
Apache License 2.0
1.81k stars 804 forks source link

ATS 9.1.4 memory leak and crashed with signal 6 #9969

Open xq2248 opened 1 year ago

xq2248 commented 1 year ago

ATS build options: /configure --prefix=/home/zxats --with-user=zxats --with-group=zxats --enable-hwloc --enable-example-plugins --enable-experimental-plugins --with-jemalloc=./jemalloc-5.2.1

Test Case: The sizes of all the cache objects in orgin server are the same, a lot of 28KB pictures. The ram_cache_cutoff is set to 1KB, so the cached pictures can be cache in RAM. With pressure， ATS crashed with 48G memory consumption. When ram_cache_cutoff is set to 2MB, the pictures are not be cached into RAM. ATS will not crash. Tt seems that memory leaking happens in ramcache, or it is a bug of jemalloc-5.2.1?

The crash log as follows: Process: [TS_MAIN] [33703] Version: Traffic Server 9.1.4 System Version: Linux x86_64 #1 SMP Sat Mar 14 01:26:23 CST 2020 4.18.0-147.3.1.el7_1.x86_64 Date: Fri, 07 Jul 2023 14:23:33 +0800

Signal Status: siginfo.si_signo: 6 (Aborted) siginfo.si_pid: 33703 siginfo.si_uid: 12352 siginfo.si_code: 0xfffffffa (-6) siginfo.si_addr: 0x00003040000083a7

CPU Registers: R8 :0x0000000000000088 R9 :0x00007fe6c066a018 R10:0x0000000000000008 R11:0x0000000000000202 R12:0x0000000000001000 R13:0x0000000000000000 R14:0x00007fe6b1e9a050 R15:0x0000000000100000 RDI:0x00000000000083a7 RSI:0x000000000000842b RBP:0x0000000000100000 RBX:0x0000000000001000 RDX:0x0000000000000006 RAX:0x0000000000000000 RCX:0x00007fe6bf205487 RSP:0x00007fe697a0beb8 RIP:0x00007fe6bf205487 EFL:0x0000000000000202 CSGSFS:0x002b000000000033 ERR:0x0000000000000000 TRAPNO:0x0000000000000000 OLDMASK:0x0000000000000000 CR2:0x0000000000000000

Unable to retrieve backtrace: [6] Error reading from socket.

Process Status: Name: [TS_MAIN] Umask: 0027 State: S (sleeping) Tgid: 33703 Ngid: 0 Pid: 33703 PPid: 33693 TracerPid: 0 Uid: 12352 12352 12352 12352 Gid: 12352 12352 12352 12352 FDSize: 32768 Groups: 6 12352 NStgid: 33703 NSpid: 33703 NSpgid: 33692 NSsid: 1330 VmPeak: 44411796 kB VmSize: 44411796 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 36940536 kB VmRSS: 36940536 kB RssAnon: 36928164 kB RssFile: 12372 kB RssShmem: 0 kB VmData: 44328600 kB VmStk: 132 kB VmExe: 3900 kB VmLib: 10416 kB VmPTE: 79432 kB VmSwap: 0 kB HugetlbPages: 0 kB CoreDumping: 0 Threads: 376 SigQ: 0/513058 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000001001 SigCgt: 0000000180004efe CapInh: 0000000000000000 CapPrm: 0000000000000400 CapEff: 0000000000000400 CapBnd: 0000003fffffffff CapAmb: 0000000000000000 NoNewPrivs: 0 Seccomp: 0 Speculation_Store_Bypass: vulnerable Cpus_allowed: ff,ffffffff Cpus_allowed_list: 0-39 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 Mems_allowed_list: 0-1 voluntary_ctxt_switches: 885 nonvoluntary_ctxt_switches: 43

Process Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 4194304 4194304 bytes Max core file size 0 0 bytes Max resident set unlimited unlimited bytes Max processes 4096 513058 processes Max open files 65535 65535 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 513058 513058 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us

Unable to retrieve Traffic Server records: [6] Error reading from socket.

osi8 commented 1 year ago

I can't say if the reason is the same (don't know, but this is the most similar issue I think), but I'm having a Signal 6 crash too. In my case it happens in the packaged version included in the three latest ubuntu version (ats 9.1.x up to 9.2.0). I have trafficserver running inside a docker container, but the problem persists even if network: host and privileged: true are set on the container. Apart from the crash log I saw the following on stdout/stderr:

Attaching to ats9
ats9    | Fatal: MIME.cc:622: failed assertion `slot_num == slot_index`
ats9    | [Jul 20 14:36:28.913] traffic_crashlo NOTE: crashlog started, target=11, debug=false syslog=true, uid=100 euid=0
ats9    | [Jul 20 14:36:28.913] traffic_crashlo WARNING: failed to initialize management API: [5] Error establishing socket connection.
ats9    | [Jul 20 14:36:28.913] traffic_crashlo NOTE: logging to 0x55c47894fc00
ats9    | [Jul 20 14:36:28.915] traffic_crashlo ERROR: wrote crash log to /var/log/trafficserver/crash-2023-07-20-143628.log
ats9    | traffic_server: received signal 6 (Aborted)
ats9    | traffic_server - STACK TRACE: 
ats9    | /lib/x86_64-linux-gnu/libc.so.6(+0x3c4b0)[0x7f437fece4b0]
ats9    | /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x11b)[0x7f437ff22ffb]
ats9    | /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x16)[0x7f437fece406]
ats9    | /lib/x86_64-linux-gnu/libc.so.6(abort+0xd7)[0x7f437feb487c]
ats9    | /usr/lib/trafficserver/libtscore.so.9(+0x4231e)[0x7f4380ab631e]
ats9    | /usr/lib/trafficserver/libtscore.so.9(+0x4233b)[0x7f4380ab633b]
ats9    | /usr/bin/traffic_server(_Z21mime_hdr_sanity_checkP11MIMEHdrImpl+0x18b)[0x559f4cdfc8fb]
ats9    | /usr/bin/traffic_server(_Z21mime_hdr_field_detachP11MIMEHdrImplP9MIMEFieldb+0x182)[0x559f4ce078b2]
ats9    | /usr/bin/traffic_server(+0x454362)[0x559f4cf94362]
ats9    | /usr/bin/traffic_server(+0x454508)[0x559f4cf94508]
ats9    | /usr/bin/traffic_server(_ZN12HttpTransact40merge_response_header_with_cached_headerEP7HTTPHdrS1_+0x3c7)[0x559f4cd1e537]
ats9    | /usr/bin/traffic_server(_ZN12HttpTransact41merge_and_update_headers_for_cache_updateEPNS_5StateE+0x1bd)[0x559f4cd1e99d]
ats9    | /usr/bin/traffic_server(_ZN12HttpTransact49handle_cache_operation_on_forward_server_responseEPNS_5StateE+0x6e9)[0x559f4cd226e9]
ats9    | /usr/bin/traffic_server(_ZN6HttpSM32call_transact_and_set_next_stateEPFvPN12HttpTransact5StateEE+0x219)[0x559f4ccefdb9]
ats9    | /usr/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x95e)[0x559f4cccd1ee]
ats9    | /usr/bin/traffic_server(_ZN6HttpSM33state_read_server_response_headerEiPv+0x41c)[0x559f4ccd083c]
ats9    | /usr/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x110)[0x559f4cccc6a0]
ats9    | /usr/bin/traffic_server(+0x3fce24)[0x559f4cf3ce24]
ats9    | /usr/bin/traffic_server(_ZN17SSLNetVConnection11net_read_ioEP10NetHandlerP7EThread+0xebe)[0x559f4cf016ae]
ats9    | /usr/bin/traffic_server(_ZN10NetHandler18process_ready_listEv+0x10c)[0x559f4cf3238c]
ats9    | /usr/bin/traffic_server(_ZN10NetHandler15waitForActivityEl+0x235)[0x559f4cf32895]
ats9    | /usr/bin/traffic_server(_ZN7EThread15execute_regularEv+0x391)[0x559f4cf73f71]
ats9    | /usr/bin/traffic_server(_ZN7EThread7executeEv+0x131)[0x559f4cf744f1]
ats9    | /usr/bin/traffic_server(+0x42bf2a)[0x559f4cf6bf2a]
ats9    | /lib/x86_64-linux-gnu/libc.so.6(+0x8f18a)[0x7f437ff2118a]
ats9    | /lib/x86_64-linux-gnu/libc.so.6(+0x11dbd0)[0x7f437ffafbd0]

crashlog: 

Process:            [TS_MAIN] [11]
File:               /usr/bin/traffic_server
Version:            Traffic Server 9.2.0
System Version:     Linux x86_64 #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 5.15.0-76-generic
Date:               Thu, 20 Jul 2023 14:36:28 +0200

No target signal information

No target CPU registers

Unable to retrieve backtrace: [5] Error establishing socket connection.

Process Status:
Name:   [TS_MAIN]
Umask:  0022
State:  S (sleeping)
Tgid:   11
Ngid:   0
Pid:    11
PPid:   1
TracerPid:      0
Uid:    100     100     100     100
Gid:    101     101     101     101
FDSize: 64
Groups: 101 
NStgid: 11
NSpid:  11
NSpgid: 1
NSsid:  1
VmPeak:  1118508 kB
VmSize:  1118508 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:     62604 kB
VmRSS:     62604 kB
RssAnon:           46728 kB
RssFile:           15876 kB
RssShmem:              0 kB
VmData:    84640 kB
VmStk:       132 kB
VmExe:      3704 kB
VmLib:     11324 kB
VmPTE:       308 kB
VmSwap:        0 kB
HugetlbPages:          0 kB
CoreDumping:    0
THP_enabled:    1
Threads:        20
SigQ:   0/15191
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000080001001
SigCgt: 0000000100004efe
CapInh: 0000000000000000
CapPrm: 000000000000540a
CapEff: 0000000000005400
CapBnd: 000001ffffffffff
CapAmb: 0000000000000000
NoNewPrivs:     0
Seccomp:        0
Seccomp_filters:        0
Speculation_Store_Bypass:       thread vulnerable
SpeculationIndirectBranch:      conditional enabled
Cpus_allowed:   3
Cpus_allowed_list:      0-1
Mems_allowed:   00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001
Mems_allowed_list:      0
voluntary_ctxt_switches:        48
nonvoluntary_ctxt_switches:     85

Process Limits:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            unlimited            unlimited            bytes     
Max core file size        unlimited            unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             unlimited            unlimited            processes 
Max open files            1048576              1048576              files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       15191                15191                signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        

Memory Regions:
559f4cb40000-559f4cc00000 r--p 00000000 00:2b 162598                     /usr/bin/traffic_server
559f4cc00000-559f4cf9e000 r-xp 000c0000 00:2b 162598                     /usr/bin/traffic_server
559f4cf9e000-559f4d075000 r--p 0045e000 00:2b 162598                     /usr/bin/traffic_server
559f4d075000-559f4d08b000 r--p 00534000 00:2b 162598                     /usr/bin/traffic_server
559f4d08b000-559f4d090000 rw-p 0054a000 00:2b 162598                     /usr/bin/traffic_server
559f4d090000-559f4d243000 rw-p 00000000 00:00 0 
559f4d48c000-559f4d6ef000 rw-p 00000000 00:00 0                          [heap]
7f4338000000-7f4338021000 rw-p 00000000 00:00 0 
7f4338021000-7f433c000000 ---p 00000000 00:00 0 
7f433c000000-7f433c040000 rw-p 00000000 00:00 0 
7f433c040000-7f4340000000 ---p 00000000 00:00 0 
7f4340000000-7f434004b000 rw-p 00000000 00:00 0 
7f434004b000-7f4344000000 ---p 00000000 00:00 0 
7f4344000000-7f4344021000 rw-p 00000000 00:00 0 
7f4344021000-7f4348000000 ---p 00000000 00:00 0 
7f4348000000-7f4348021000 rw-p 00000000 00:00 0 
7f4348021000-7f434c000000 ---p 00000000 00:00 0 
7f434c000000-7f434c021000 rw-p 00000000 00:00 0 
7f434c021000-7f4350000000 ---p 00000000 00:00 0 
7f4350000000-7f4350021000 rw-p 00000000 00:00 0 
7f4350021000-7f4354000000 ---p 00000000 00:00 0 
7f4354000000-7f4354021000 rw-p 00000000 00:00 0 
7f4354021000-7f4358000000 ---p 00000000 00:00 0 
7f4358000000-7f4358021000 rw-p 00000000 00:00 0 
7f4358021000-7f435c000000 ---p 00000000 00:00 0 
7f435c000000-7f435c021000 rw-p 00000000 00:00 0 
7f435c021000-7f4360000000 ---p 00000000 00:00 0 
7f4360000000-7f4360021000 rw-p 00000000 00:00 0 
7f4360021000-7f4364000000 ---p 00000000 00:00 0 
7f4368000000-7f4368021000 rw-p 00000000 00:00 0 
7f4368021000-7f436c000000 ---p 00000000 00:00 0 
7f436d065000-7f436d066000 rw-p 00000000 00:00 0 
7f436d066000-7f436d466000 rw-p 00000000 00:00 0 
7f436d466000-7f436d468000 rw-p 00000000 00:00 0 
7f436d468000-7f436d668000 rw-p 00000000 00:00 0 
7f436d668000-7f436d791000 rw-p 00000000 00:00 0 
7f436d791000-7f436d811000 rw-p 00000000 00:00 0 
7f436d811000-7f436d813000 rw-p 00000000 00:00 0 
7f436d813000-7f436d913000 rw-p 00000000 00:00 0 
7f436d913000-7f436d914000 rw-p 00000000 00:00 0 
7f436d914000-7f436d915000 ---p 00000000 00:00 0 
7f436d915000-7f436da15000 rw-p 00000000 00:00 0 
7f436da15000-7f436da16000 ---p 00000000 00:00 0 
7f436da16000-7f436dd30000 rw-p 00000000 00:00 0 
7f436dd30000-7f436dd31000 ---p 00000000 00:00 0 
7f436dd31000-7f436df3e000 rw-p 00000000 00:00 0 
7f436df3e000-7f436df3f000 ---p 00000000 00:00 0 
7f436df3f000-7f436e14c000 rw-p 00000000 00:00 0 
7f436e14c000-7f436e14d000 ---p 00000000 00:00 0 
7f436e14d000-7f436e35a000 rw-p 00000000 00:00 0 
7f436e35a000-7f436e35b000 ---p 00000000 00:00 0 
7f436e35b000-7f436f19e000 rw-p 00000000 00:00 0 
7f436f19e000-7f436f19f000 ---p 00000000 00:00 0 
7f436f19f000-7f436f3ac000 rw-p 00000000 00:00 0 
7f436f3ac000-7f436f3ad000 ---p 00000000 00:00 0 
7f436f3ad000-7f436f5ba000 rw-p 00000000 00:00 0 
7f436f5ba000-7f436f5bb000 ---p 00000000 00:00 0 
7f436f5bb000-7f436f7c8000 rw-p 00000000 00:00 0 
7f436f7c8000-7f436f7c9000 ---p 00000000 00:00 0 
7f436f7c9000-7f436f9d6000 rw-p 00000000 00:00 0 
7f436f9d6000-7f436f9d7000 ---p 00000000 00:00 0 
7f436f9d7000-7f436fbe4000 rw-p 00000000 00:00 0 
7f436fbe4000-7f436fbe5000 ---p 00000000 00:00 0 
7f436fbe5000-7f436fdf2000 rw-p 00000000 00:00 0 
7f436fdf2000-7f436fdf3000 ---p 00000000 00:00 0 
7f436fdf3000-7f437009f000 rw-p 00000000 00:00 0 
7f437009f000-7f4374000000 ---p 00000000 00:00 0 
7f4374000000-7f4374030000 rw-p 00000000 00:00 0 
7f4374030000-7f4374034000 rw-p 00000000 00:00 0 
7f4374034000-7f4374170000 rw-p 00000000 00:00 0 
7f4374170000-7f43741b0000 rw-p 00000000 00:00 0 
7f43741b0000-7f4374229000 rw-p 00000000 00:00 0 
7f4374229000-7f4378000000 ---p 00000000 00:00 0 
7f4378000000-7f437803d000 rw-p 00000000 00:00 0 
7f437803d000-7f437c000000 ---p 00000000 00:00 0 
7f437c012000-7f437c015000 r--p 00000000 00:2b 162649                     /usr/lib/trafficserver/modules/regex_remap.so
7f437c015000-7f437c01a000 r-xp 00003000 00:2b 162649                     /usr/lib/trafficserver/modules/regex_remap.so
7f437c01a000-7f437c01b000 r--p 00008000 00:2b 162649                     /usr/lib/trafficserver/modules/regex_remap.so
7f437c01b000-7f437c01c000 r--p 00009000 00:2b 162649                     /usr/lib/trafficserver/modules/regex_remap.so
7f437c01c000-7f437c01d000 rw-p 0000a000 00:2b 162649                     /usr/lib/trafficserver/modules/regex_remap.so
7f437c01d000-7f437c01e000 ---p 00000000 00:00 0 
7f437c01e000-7f437c22b000 rw-p 00000000 00:00 0 
7f437c22b000-7f437c22c000 r--p 00000000 00:2b 162686                     /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9
7f437c22c000-7f437c22d000 r-xp 00001000 00:2b 162686                     /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9
7f437c22d000-7f437c24c000 r--p 00002000 00:2b 162686                     /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9
7f437c24c000-7f437c24d000 r--p 00020000 00:2b 162686                     /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9
7f437c24d000-7f437c24e000 rw-p 00021000 00:2b 162686                     /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9
7f437c24e000-7f437c24f000 r--p 00000000 00:2b 162690                     /usr/lib/x86_64-linux-gnu/libbrotlienc.so.1.0.9
7f437c24f000-7f437c28c000 r-xp 00001000 00:2b 162690                     /usr/lib/x86_64-linux-gnu/libbrotlienc.so.1.0.9
7f437c28c000-7f437c2dd000 r--p 0003e000 00:2b 162690                     /usr/lib/x86_64-linux-gnu/libbrotlienc.so.1.0.9
7f437c2dd000-7f437c2de000 r--p 0008e000 00:2b 162690                     /usr/lib/x86_64-linux-gnu/libbrotlienc.so.1.0.9
7f437c2de000-7f437c2df000 rw-p 0008f000 00:2b 162690                     /usr/lib/x86_64-linux-gnu/libbrotlienc.so.1.0.9
7f437c2df000-7f437c2e1000 r--p 00000000 00:2b 162654                     /usr/lib/trafficserver/modules/stats_over_http.so
7f437c2e1000-7f437c2e4000 r-xp 00002000 00:2b 162654                     /usr/lib/trafficserver/modules/stats_over_http.so
7f437c2e4000-7f437c2e5000 r--p 00005000 00:2b 162654                     /usr/lib/trafficserver/modules/stats_over_http.so
7f437c2e5000-7f437c2e6000 r--p 00006000 00:2b 162654                     /usr/lib/trafficserver/modules/stats_over_http.so
7f437c2e6000-7f437c2e7000 rw-p 00007000 00:2b 162654                     /usr/lib/trafficserver/modules/stats_over_http.so
7f437c2e7000-7f437c388000 rw-p 00000000 00:00 0 
7f437c388000-7f437c389000 ---p 00000000 00:00 0 
7f437c389000-7f437c596000 rw-p 00000000 00:00 0 
7f437c596000-7f437c597000 ---p 00000000 00:00 0 
7f437c597000-7f437c7a5000 rw-p 00000000 00:00 0 
7f437c7a5000-7f437c825000 rw-p 00000000 00:00 0 
7f437c825000-7f437c9e0000 rw-p 00000000 00:00 0 
7f437c9e0000-7f437c9e1000 ---p 00000000 00:00 0 
7f437c9e1000-7f437cae1000 rw-p 00000000 00:00 0 
7f437cae1000-7f437cae2000 ---p 00000000 00:00 0 
7f437cae2000-7f437cbe2000 rw-p 00000000 00:00 0 
7f437cbe2000-7f437cbe3000 ---p 00000000 00:00 0 
7f437cbe3000-7f437d10a000 rw-p 00000000 00:00 0 
7f437d10a000-7f437d12a000 rwxp 00000000 00:00 0 
7f437d12a000-7f437d237000 rw-p 00000000 00:00 0 
7f437d237000-7f437d238000 ---p 00000000 00:00 0 
7f437d238000-7f437daa9000 rw-p 00000000 00:00 0 
7f437daa9000-7f437daab000 r--p 00000000 00:2b 159440                     /usr/lib/x86_64-linux-gnu/libmd.so.0.0.5
7f437daab000-7f437dab2000 r-xp 00002000 00:2b 159440                     /usr/lib/x86_64-linux-gnu/libmd.so.0.0.5
7f437dab2000-7f437dab4000 r--p 00009000 00:2b 159440                     /usr/lib/x86_64-linux-gnu/libmd.so.0.0.5
7f437dab4000-7f437dab5000 r--p 0000a000 00:2b 159440                     /usr/lib/x86_64-linux-gnu/libmd.so.0.0.5
7f437dab5000-7f437dab6000 rw-p 0000b000 00:2b 159440                     /usr/lib/x86_64-linux-gnu/libmd.so.0.0.5
7f437dab6000-7f437daba000 r--p 00000000 00:2b 162692                     /usr/lib/x86_64-linux-gnu/libbsd.so.0.11.7
7f437daba000-7f437dac5000 r-xp 00004000 00:2b 162692                     /usr/lib/x86_64-linux-gnu/libbsd.so.0.11.7
7f437dac5000-7f437dac8000 r--p 0000f000 00:2b 162692                     /usr/lib/x86_64-linux-gnu/libbsd.so.0.11.7
7f437dac8000-7f437dac9000 r--p 00012000 00:2b 162692                     /usr/lib/x86_64-linux-gnu/libbsd.so.0.11.7
7f437dac9000-7f437daca000 rw-p 00013000 00:2b 162692                     /usr/lib/x86_64-linux-gnu/libbsd.so.0.11.7
7f437daca000-7f437dacb000 rw-p 00000000 00:00 0 
7f437dacb000-7f437dacd000 r--p 00000000 00:2b 162682                     /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7f437dacd000-7f437dacf000 r-xp 00002000 00:2b 162682                     /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7f437dacf000-7f437dad1000 r--p 00004000 00:2b 162682                     /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7f437dad1000-7f437dad2000 r--p 00005000 00:2b 162682                     /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7f437dad2000-7f437dad3000 rw-p 00006000 00:2b 162682                     /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7f437dad3000-7f437dad4000 r--p 00000000 00:2b 162680                     /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7f437dad4000-7f437dad6000 r-xp 00001000 00:2b 162680                     /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7f437dad6000-7f437dad7000 r--p 00003000 00:2b 162680                     /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7f437dad7000-7f437dad8000 r--p 00003000 00:2b 162680                     /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7f437dad8000-7f437dad9000 rw-p 00004000 00:2b 162680                     /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7f437dad9000-7f437dae4000 r--p 00000000 00:2b 162759                     /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7f437dae4000-7f437daf8000 r-xp 0000b000 00:2b 162759                     /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7f437daf8000-7f437db01000 r--p 0001f000 00:2b 162759                     /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7f437db01000-7f437db02000 r--p 00027000 00:2b 162759                     /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7f437db02000-7f437db03000 rw-p 00028000 00:2b 162759                     /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7f437db03000-7f437db07000 r--p 00000000 00:2b 162684                     /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7f437db07000-7f437db12000 r-xp 00004000 00:2b 162684                     /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7f437db12000-7f437db16000 r--p 0000f000 00:2b 162684                     /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7f437db16000-7f437db17000 r--p 00012000 00:2b 162684                     /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7f437db17000-7f437db18000 rw-p 00013000 00:2b 162684                     /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7f437db18000-7f437db31000 r--p 00000000 00:2b 162676                     /usr/lib/x86_64-linux-gnu/libX11.so.6.4.0
7f437db31000-7f437dbbc000 r-xp 00019000 00:2b 162676                     /usr/lib/x86_64-linux-gnu/libX11.so.6.4.0
7f437dbbc000-7f437dc51000 r--p 000a4000 00:2b 162676                     /usr/lib/x86_64-linux-gnu/libX11.so.6.4.0
7f437dc51000-7f437dc52000 r--p 00139000 00:2b 162676                     /usr/lib/x86_64-linux-gnu/libX11.so.6.4.0
7f437dc52000-7f437dc56000 rw-p 0013a000 00:2b 162676                     /usr/lib/x86_64-linux-gnu/libX11.so.6.4.0
7f437dc56000-7f437dc58000 r--p 00000000 00:2b 162678                     /usr/lib/x86_64-linux-gnu/libXNVCtrl.so.0.0.0
7f437dc58000-7f437dc5c000 r-xp 00002000 00:2b 162678                     /usr/lib/x86_64-linux-gnu/libXNVCtrl.so.0.0.0
7f437dc5c000-7f437dc5d000 r--p 00006000 00:2b 162678                     /usr/lib/x86_64-linux-gnu/libXNVCtrl.so.0.0.0
7f437dc5d000-7f437dc5e000 r--p 00006000 00:2b 162678                     /usr/lib/x86_64-linux-gnu/libXNVCtrl.so.0.0.0
7f437dc5e000-7f437dc5f000 rw-p 00007000 00:2b 162678                     /usr/lib/x86_64-linux-gnu/libXNVCtrl.so.0.0.0
7f437dc5f000-7f437dc60000 r--p 00000000 00:2b 162664                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_gl.so
7f437dc60000-7f437dc61000 r-xp 00001000 00:2b 162664                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_gl.so
7f437dc61000-7f437dc62000 r--p 00002000 00:2b 162664                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_gl.so
7f437dc62000-7f437dc63000 r--p 00002000 00:2b 162664                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_gl.so
7f437dc63000-7f437dc64000 rw-p 00003000 00:2b 162664                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_gl.so
7f437dc64000-7f437dc66000 r--p 00000000 00:2b 162735                     /usr/lib/x86_64-linux-gnu/libpciaccess.so.0.11.1
7f437dc66000-7f437dc6b000 r-xp 00002000 00:2b 162735                     /usr/lib/x86_64-linux-gnu/libpciaccess.so.0.11.1
7f437dc6b000-7f437dc6d000 r--p 00007000 00:2b 162735                     /usr/lib/x86_64-linux-gnu/libpciaccess.so.0.11.1
7f437dc6d000-7f437dc6e000 r--p 00008000 00:2b 162735                     /usr/lib/x86_64-linux-gnu/libpciaccess.so.0.11.1
7f437dc6e000-7f437dc6f000 rw-p 00009000 00:2b 162735                     /usr/lib/x86_64-linux-gnu/libpciaccess.so.0.11.1
7f437dc6f000-7f437dc71000 r--p 00000000 00:2b 162666                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_pci.so
7f437dc71000-7f437dc73000 r-xp 00002000 00:2b 162666                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_pci.so
7f437dc73000-7f437dc74000 r--p 00004000 00:2b 162666                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_pci.so
7f437dc74000-7f437dc75000 r--p 00004000 00:2b 162666                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_pci.so
7f437dc75000-7f437dc76000 rw-p 00005000 00:2b 162666                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_pci.so
7f437dc76000-7f437dc77000 r--p 00000000 00:2b 162705                     /usr/lib/x86_64-linux-gnu/libicudata.so.72.1
7f437dc77000-7f437dc78000 r-xp 00001000 00:2b 162705                     /usr/lib/x86_64-linux-gnu/libicudata.so.72.1
7f437dc78000-7f437fa46000 r--p 00002000 00:2b 162705                     /usr/lib/x86_64-linux-gnu/libicudata.so.72.1
7f437fa46000-7f437fa47000 r--p 01dcf000 00:2b 162705                     /usr/lib/x86_64-linux-gnu/libicudata.so.72.1
7f437fa47000-7f437fa48000 rw-p 01dd0000 00:2b 162705                     /usr/lib/x86_64-linux-gnu/libicudata.so.72.1
7f437fa48000-7f437faae000 r--p 00000000 00:2b 162715                     /usr/lib/x86_64-linux-gnu/libicuuc.so.72.1
7f437faae000-7f437fba3000 r-xp 00066000 00:2b 162715                     /usr/lib/x86_64-linux-gnu/libicuuc.so.72.1
7f437fba3000-7f437fc31000 r--p 0015b000 00:2b 162715                     /usr/lib/x86_64-linux-gnu/libicuuc.so.72.1
7f437fc31000-7f437fc44000 r--p 001e9000 00:2b 162715                     /usr/lib/x86_64-linux-gnu/libicuuc.so.72.1
7f437fc44000-7f437fc45000 rw-p 001fc000 00:2b 162715                     /usr/lib/x86_64-linux-gnu/libicuuc.so.72.1
7f437fc45000-7f437fc47000 rw-p 00000000 00:00 0 
7f437fc47000-7f437fc76000 r--p 00000000 00:2b 162761                     /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14
7f437fc76000-7f437fdd2000 r-xp 0002f000 00:2b 162761                     /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14
7f437fdd2000-7f437fe26000 r--p 0018b000 00:2b 162761                     /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14
7f437fe26000-7f437fe2f000 r--p 001df000 00:2b 162761                     /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14
7f437fe2f000-7f437fe30000 rw-p 001e8000 00:2b 162761                     /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.14
7f437fe30000-7f437fe31000 rw-p 00000000 00:00 0 
7f437fe31000-7f437fe33000 r--p 00000000 00:2b 162667                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_xml_libxml.so
7f437fe33000-7f437fe35000 r-xp 00002000 00:2b 162667                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_xml_libxml.so
7f437fe35000-7f437fe36000 r--p 00004000 00:2b 162667                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_xml_libxml.so
7f437fe36000-7f437fe37000 r--p 00004000 00:2b 162667                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_xml_libxml.so
7f437fe37000-7f437fe38000 rw-p 00005000 00:2b 162667                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_xml_libxml.so
7f437fe38000-7f437fe3e000 r--p 00000000 00:2b 162674                     /usr/lib/x86_64-linux-gnu/libOpenCL.so.1.0.0
7f437fe3e000-7f437fe44000 r-xp 00006000 00:2b 162674                     /usr/lib/x86_64-linux-gnu/libOpenCL.so.1.0.0
7f437fe44000-7f437fe48000 r--p 0000c000 00:2b 162674                     /usr/lib/x86_64-linux-gnu/libOpenCL.so.1.0.0
7f437fe48000-7f437fe49000 r--p 0000f000 00:2b 162674                     /usr/lib/x86_64-linux-gnu/libOpenCL.so.1.0.0
7f437fe49000-7f437fe4a000 rw-p 00010000 00:2b 162674                     /usr/lib/x86_64-linux-gnu/libOpenCL.so.1.0.0
7f437fe4a000-7f437fe4b000 r--p 00000000 00:2b 162665                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_opencl.so
7f437fe4b000-7f437fe4c000 r-xp 00001000 00:2b 162665                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_opencl.so
7f437fe4c000-7f437fe4d000 r--p 00002000 00:2b 162665                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_opencl.so
7f437fe4d000-7f437fe4e000 r--p 00002000 00:2b 162665                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_opencl.so
7f437fe4e000-7f437fe4f000 rw-p 00003000 00:2b 162665                     /usr/lib/x86_64-linux-gnu/hwloc/hwloc_opencl.so
7f437fe4f000-7f437fe58000 rw-p 00000000 00:00 0 
7f437fe58000-7f437fe5d000 r--p 00000000 00:2b 159523                     /usr/lib/x86_64-linux-gnu/libudev.so.1.7.5
7f437fe5d000-7f437fe7a000 r-xp 00005000 00:2b 159523                     /usr/lib/x86_64-linux-gnu/libudev.so.1.7.5
7f437fe7a000-7f437fe84000 r--p 00022000 00:2b 159523                     /usr/lib/x86_64-linux-gnu/libudev.so.1.7.5
7f437fe84000-7f437fe85000 r--p 0002c000 00:2b 159523                     /usr/lib/x86_64-linux-gnu/libudev.so.1.7.5
7f437fe85000-7f437fe86000 rw-p 0002d000 00:2b 159523                     /usr/lib/x86_64-linux-gnu/libudev.so.1.7.5
7f437fe86000-7f437fe89000 r--p 00000000 00:2b 159290                     /usr/lib/x86_64-linux-gnu/libcap.so.2.66
7f437fe89000-7f437fe8e000 r-xp 00003000 00:2b 159290                     /usr/lib/x86_64-linux-gnu/libcap.so.2.66
7f437fe8e000-7f437fe90000 r--p 00008000 00:2b 159290                     /usr/lib/x86_64-linux-gnu/libcap.so.2.66
7f437fe90000-7f437fe91000 r--p 0000a000 00:2b 159290                     /usr/lib/x86_64-linux-gnu/libcap.so.2.66
7f437fe91000-7f437fe92000 rw-p 0000b000 00:2b 159290                     /usr/lib/x86_64-linux-gnu/libcap.so.2.66
7f437fe92000-7f437feb4000 r--p 00000000 00:2b 159276                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f437feb4000-7f438002c000 r-xp 00022000 00:2b 159276                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f438002c000-7f4380084000 r--p 0019a000 00:2b 159276                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f4380084000-7f4380088000 r--p 001f1000 00:2b 159276                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f4380088000-7f438008a000 rw-p 001f5000 00:2b 159276                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f438008a000-7f4380097000 rw-p 00000000 00:00 0 
7f4380097000-7f438009a000 r--p 00000000 00:2b 159370                     /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f438009a000-7f43800b5000 r-xp 00003000 00:2b 159370                     /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f43800b5000-7f43800b9000 r--p 0001e000 00:2b 159370                     /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f43800b9000-7f43800ba000 r--p 00021000 00:2b 159370                     /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f43800ba000-7f43800bb000 rw-p 00022000 00:2b 159370                     /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
7f43800bb000-7f43800bd000 rw-p 00000000 00:00 0 
7f43800bd000-7f43800cb000 r--p 00000000 00:2b 159436                     /usr/lib/x86_64-linux-gnu/libm.so.6
7f43800cb000-7f4380149000 r-xp 0000e000 00:2b 159436                     /usr/lib/x86_64-linux-gnu/libm.so.6
7f4380149000-7f43801a4000 r--p 0008c000 00:2b 159436                     /usr/lib/x86_64-linux-gnu/libm.so.6
7f43801a4000-7f43801a5000 r--p 000e6000 00:2b 159436                     /usr/lib/x86_64-linux-gnu/libm.so.6
7f43801a5000-7f43801a6000 rw-p 000e7000 00:2b 159436                     /usr/lib/x86_64-linux-gnu/libm.so.6
7f43801a6000-7f4380242000 r--p 00000000 00:2b 159506                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.31
7f4380242000-7f4380371000 r-xp 0009c000 00:2b 159506                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.31
7f4380371000-7f43803fe000 r--p 001cb000 00:2b 159506                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.31
7f43803fe000-7f4380409000 r--p 00257000 00:2b 159506                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.31
7f4380409000-7f438040c000 rw-p 00262000 00:2b 159506                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.31
7f438040c000-7f4380410000 rw-p 00000000 00:00 0 
7f4380410000-7f4380418000 r--p 00000000 00:2b 162763                     /usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.7.0
7f4380418000-7f4380441000 r-xp 00008000 00:2b 162763                     /usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.7.0
7f4380441000-7f438044b000 r--p 00031000 00:2b 162763                     /usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.7.0
7f438044b000-7f438044c000 ---p 0003b000 00:2b 162763                     /usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.7.0
7f438044c000-7f438044d000 r--p 0003b000 00:2b 162763                     /usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.7.0
7f438044d000-7f438044e000 rw-p 0003c000 00:2b 162763                     /usr/lib/x86_64-linux-gnu/libyaml-cpp.so.0.7.0
7f438044e000-7f438044f000 rw-p 00000000 00:00 0 
7f438044f000-7f4380501000 r--p 00000000 00:2b 162693                     /usr/lib/x86_64-linux-gnu/libcrypto.so.3
7f4380501000-7f438076f000 r-xp 000b2000 00:2b 162693                     /usr/lib/x86_64-linux-gnu/libcrypto.so.3
7f438076f000-7f4380840000 r--p 00320000 00:2b 162693                     /usr/lib/x86_64-linux-gnu/libcrypto.so.3
7f4380840000-7f438089b000 r--p 003f0000 00:2b 162693                     /usr/lib/x86_64-linux-gnu/libcrypto.so.3
7f438089b000-7f438089e000 rw-p 0044b000 00:2b 162693                     /usr/lib/x86_64-linux-gnu/libcrypto.so.3
7f438089e000-7f43808a1000 rw-p 00000000 00:00 0 
7f43808a1000-7f43808bf000 r--p 00000000 00:2b 162749                     /usr/lib/x86_64-linux-gnu/libssl.so.3
7f43808bf000-7f438091c000 r-xp 0001e000 00:2b 162749                     /usr/lib/x86_64-linux-gnu/libssl.so.3
7f438091c000-7f4380938000 r--p 0007b000 00:2b 162749                     /usr/lib/x86_64-linux-gnu/libssl.so.3
7f4380938000-7f4380942000 r--p 00096000 00:2b 162749                     /usr/lib/x86_64-linux-gnu/libssl.so.3
7f4380942000-7f4380946000 rw-p 000a0000 00:2b 162749                     /usr/lib/x86_64-linux-gnu/libssl.so.3
7f4380946000-7f4380949000 r--p 00000000 00:2b 159434                     /usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1
7f4380949000-7f438096a000 r-xp 00003000 00:2b 159434                     /usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1
7f438096a000-7f4380976000 r--p 00024000 00:2b 159434                     /usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1
7f4380976000-7f4380977000 r--p 00030000 00:2b 159434                     /usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1
7f4380977000-7f4380978000 rw-p 00031000 00:2b 159434                     /usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1
7f4380978000-7f438097a000 rw-p 00000000 00:00 0 
7f438097a000-7f438097d000 r--p 00000000 00:2b 159536                     /usr/lib/x86_64-linux-gnu/libz.so.1.2.13
7f438097d000-7f438098f000 r-xp 00003000 00:2b 159536                     /usr/lib/x86_64-linux-gnu/libz.so.1.2.13
7f438098f000-7f4380996000 r--p 00015000 00:2b 159536                     /usr/lib/x86_64-linux-gnu/libz.so.1.2.13
7f4380996000-7f4380997000 r--p 0001b000 00:2b 159536                     /usr/lib/x86_64-linux-gnu/libz.so.1.2.13
7f4380997000-7f4380998000 rw-p 0001c000 00:2b 159536                     /usr/lib/x86_64-linux-gnu/libz.so.1.2.13
7f4380998000-7f438099a000 r--p 00000000 00:2b 162737                     /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3
7f438099a000-7f43809f1000 r-xp 00002000 00:2b 162737                     /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3
7f43809f1000-7f4380a0e000 r--p 00059000 00:2b 162737                     /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3
7f4380a0e000-7f4380a0f000 r--p 00075000 00:2b 162737                     /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3
7f4380a0f000-7f4380a10000 rw-p 00076000 00:2b 162737                     /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3
7f4380a10000-7f4380a16000 r--p 00000000 00:2b 162703                     /usr/lib/x86_64-linux-gnu/libhwloc.so.15.6.1
7f4380a16000-7f4380a58000 r-xp 00006000 00:2b 162703                     /usr/lib/x86_64-linux-gnu/libhwloc.so.15.6.1
7f4380a58000-7f4380a6b000 r--p 00048000 00:2b 162703                     /usr/lib/x86_64-linux-gnu/libhwloc.so.15.6.1
7f4380a6b000-7f4380a6c000 r--p 0005b000 00:2b 162703                     /usr/lib/x86_64-linux-gnu/libhwloc.so.15.6.1
7f4380a6c000-7f4380a6d000 rw-p 0005c000 00:2b 162703                     /usr/lib/x86_64-linux-gnu/libhwloc.so.15.6.1
7f4380a6f000-7f4380a70000 r--p 00000000 00:2b 162628                     /usr/lib/trafficserver/libtscpputil.so.9.2.0
7f4380a70000-7f4380a71000 r-xp 00001000 00:2b 162628                     /usr/lib/trafficserver/libtscpputil.so.9.2.0
7f4380a71000-7f4380a72000 r--p 00002000 00:2b 162628                     /usr/lib/trafficserver/libtscpputil.so.9.2.0
7f4380a72000-7f4380a73000 r--p 00002000 00:2b 162628                     /usr/lib/trafficserver/libtscpputil.so.9.2.0
7f4380a73000-7f4380a74000 rw-p 00003000 00:2b 162628                     /usr/lib/trafficserver/libtscpputil.so.9.2.0
7f4380a74000-7f4380a8f000 r--p 00000000 00:2b 162624                     /usr/lib/trafficserver/libtscore.so.9.2.0
7f4380a8f000-7f4380ae4000 r-xp 0001b000 00:2b 162624                     /usr/lib/trafficserver/libtscore.so.9.2.0
7f4380ae4000-7f4380af7000 r--p 00070000 00:2b 162624                     /usr/lib/trafficserver/libtscore.so.9.2.0
7f4380af7000-7f4380afa000 r--p 00083000 00:2b 162624                     /usr/lib/trafficserver/libtscore.so.9.2.0
7f4380afa000-7f4380afc000 rw-p 00086000 00:2b 162624                     /usr/lib/trafficserver/libtscore.so.9.2.0
7f4380afc000-7f4380b00000 rw-p 00000000 00:00 0 
7f4380b00000-7f4380b01000 r--p 00000000 00:2b 159240                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f4380b01000-7f4380b29000 r-xp 00001000 00:2b 159240                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f4380b29000-7f4380b33000 r--p 00029000 00:2b 159240                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f4380b33000-7f4380b35000 r--p 00033000 00:2b 159240                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f4380b35000-7f4380b37000 rw-p 00035000 00:2b 159240                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7fff722b9000-7fff722da000 rw-p 00000000 00:00 0                          [stack]
7fff7238e000-7fff72392000 r--p 00000000 00:00 0                          [vvar]
7fff72392000-7fff72394000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

Unable to retrieve Traffic Server records: [5] Error establishing socket connection.

Is there anything I could do/provide to help solving this problem? I can reproduce the error by repeatedly requesting the same m3u8 playlist, which in turn gets retrieved from the origin. It's caching duration is only 2 seconds, since it is for a live stream. Upstream is an Nginx.

osi8 commented 1 year ago

I just installed trafficserver 9.2.0 from ubuntu 23.04 repos directly on the host (no containers in between) and reproduced the issue:

==> /var/log/trafficserver/manager.log <==
[Jul 20 16:15:49.493] traffic_manager NOTE: [LocalManager::pollMgmtProcessServer] Server Process terminated due to Sig 6: Aborted
[Jul 20 16:15:49.493] traffic_manager NOTE: [Alarms::signalAlarm] Server Process was reset
[Jul 20 16:15:50.495] traffic_manager NOTE: Relaunching proxy after 2 sec..
[Jul 20 16:15:52.495] traffic_manager NOTE: [ProxyStateSet] Traffic Server Args: ' -M'
[Jul 20 16:15:52.496] traffic_manager NOTE: [LocalManager::listenForProxy] Listening on port: 80 (ipv4)
[Jul 20 16:15:52.496] traffic_manager NOTE: [LocalManager::listenForProxy] Listening on port: 80 (ipv6)
[Jul 20 16:15:52.496] traffic_manager NOTE: [LocalManager::listenForProxy] Listening on port: 443 (ipv4)
[Jul 20 16:15:52.496] traffic_manager NOTE: [LocalManager::listenForProxy] Listening on port: 443 (ipv6)
[Jul 20 16:15:52.496] traffic_manager NOTE: [LocalManager::startProxy] Launching ts process
[Jul 20 16:15:52.513] traffic_manager NOTE: [LocalManager::pollMgmtProcessServer] New process connecting fd '14'
[Jul 20 16:15:52.513] traffic_manager NOTE: [Alarms::signalAlarm] Server Process born

==> /var/log/trafficserver/diags.log <==
[Jul 20 16:15:54.526] traffic_server STATUS: opened /var/log/trafficserver/diags.log
[Jul 20 16:15:54.526] traffic_server NOTE: updated diags config
[Jul 20 16:15:54.538] traffic_server NOTE: storage.config loading ...
[Jul 20 16:15:54.538] traffic_server NOTE: storage.config finished loading
[Jul 20 16:15:54.540] traffic_server NOTE: ip_allow.yaml loading ...
[Jul 20 16:15:54.540] traffic_server NOTE: ip_allow.yaml finished loading
[Jul 20 16:15:54.540] traffic_server NOTE: parent.config loading ...
[Jul 20 16:15:54.541] traffic_server NOTE: parent.config finished loading
[Jul 20 16:15:54.541] traffic_server NOTE: /etc/trafficserver/logging.yaml loading ...
[Jul 20 16:15:54.541] traffic_server NOTE: /etc/trafficserver/logging.yaml finished loading
[Jul 20 16:15:54.542] traffic_server NOTE: logging initialized[3], logging_mode = 3
[Jul 20 16:15:54.542] traffic_server NOTE: Initialized plugin_dynamic_reload_mode: 0
[Jul 20 16:15:54.543] traffic_server NOTE: plugin.config loading ...
[Jul 20 16:15:54.543] traffic_server NOTE: loading plugin '/usr/lib/trafficserver/modules/stats_over_http.so'
[Jul 20 16:15:54.543] traffic_server NOTE: plugin.config finished loading
[Jul 20 16:15:54.544] traffic_server NOTE: /etc/trafficserver/sni.yaml loading ...
[Jul 20 16:15:54.544] traffic_server NOTE: /etc/trafficserver/sni.yaml finished loading
[Jul 20 16:15:54.544] traffic_server NOTE: ssl_multicert.config loading ...
[Jul 20 16:15:54.572] traffic_server NOTE: /etc/trafficserver/ssl_multicert.config finished loading
[Jul 20 16:15:54.572] traffic_server NOTE: volume.config loading ...
[Jul 20 16:15:54.572] traffic_server NOTE: volume.config finished loading
[Jul 20 16:15:54.577] traffic_server NOTE: remap.config loading ...
[Jul 20 16:15:54.577] traffic_server NOTE: strategies.yaml loading ...
[Jul 20 16:15:54.577] traffic_server NOTE: No NextHop strategy configs were loaded.
[Jul 20 16:15:54.577] traffic_server NOTE: strategies.yaml finished loading
[Jul 20 16:15:54.580] traffic_server NOTE: remap.config finished loading
[Jul 20 16:15:54.587] [TS_MAIN] NOTE: traffic server running
[Jul 20 16:15:54.600] [ET_NET 0] NOTE: recovery clearing offsets of Vol /var/cache/trafficserver/cache.db 16384:786430 : [16142336, 24530944] sync_serial 3 next 4
[Jul 20 16:15:54.607] [ET_NET 0] NOTE: hosting.config loading ...
[Jul 20 16:15:54.608] [ET_NET 0] NOTE: hosting.config finished loading
[Jul 20 16:15:54.610] [ET_NET 0] NOTE: cache enabled
[Jul 20 16:15:54.610] [ET_NET 0] NOTE: Traffic Server is fully initialized.
osi8 commented 1 year ago

We returned to Docker based (base os ubuntu 20.04) trafficserver in version 8.0.5 (packaged version from ubuntu) where the error does not appear. But since this version is EOL we would rather use a stable and supported version.

github-actions[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.