search

ossrs / srs

SRS is a simple, high-efficiency, real-time video server supporting RTMP, WebRTC, HLS, HTTP-FLV, SRT, MPEG-DASH, and GB28181.
https://ossrs.io
MIT License
24.72k stars 5.28k forks source link

API: The VmSize of SRS is huge and incorrect, if enabled asan. #4041

Open winlinvip opened 2 months ago

winlinvip commented 2 months ago

Describe the bug When started SRS, the VmSize is huge and incorrect. You can get the VmSize by top or ps.

Version SRS 5 or 6.

To Reproduce Steps to reproduce the behavior:

  1. Run SRS by ./objs/srs -c conf/console.conf
  2. Run command top -p $(pidof srs)
  3. Or run command by ps -u -p $(pidof srs)
  4. The VmSize is huge, about 20TB.

Expected behavior Should show correct VmSize.

Screenshots

ps -u -p $(pidof srs)

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     3151451  2.6 11.6 21474955020 237412 pts/0 Sl+ 10:07   0:07 ./objs/srs -c conf/console.conf
top -p $(pidof srs)

top - 10:13:14 up 144 days, 23:11,  2 users,  load average: 0.41, 0.57, 0.72
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  3.3 us,  6.7 sy,  0.0 ni, 90.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   1987.4 total,     96.3 free,   1385.2 used,    505.9 buff/cache
MiB Swap:   1025.0 total,    642.2 free,    382.8 used.    426.5 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                                        
3151451 root      20   0   20.0t 238052  19936 S   0.0  11.7   0:07.93 srs

Additional context Maybe caused by custom stack for coroutine.

winlinvip commented 2 months ago

Get the status of SRS:

top -p $(pidof srs) ``` top - 10:13:14 up 144 days, 23:11, 2 users, load average: 0.41, 0.57, 0.72 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 3.3 us, 6.7 sy, 0.0 ni, 90.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 1987.4 total, 96.3 free, 1385.2 used, 505.9 buff/cache MiB Swap: 1025.0 total, 642.2 free, 382.8 used. 426.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3151451 root 20 0 20.0t 238052 19936 S 0.0 11.7 0:07.93 srs ```
cat /proc/$(pidof srs)/status ``` Name: srs Umask: 0022 State: S (sleeping) Tgid: 3151451 Ngid: 0 Pid: 3151451 PPid: 3113227 TracerPid: 0 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 256 Groups: 0 NStgid: 3151451 NSpid: 3151451 NSpgid: 3151451 NSsid: 3113227 VmPeak: 21474956980 kB VmSize: 21474956980 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 256512 kB VmRSS: 256512 kB RssAnon: 236576 kB RssFile: 19936 kB RssShmem: 0 kB VmData: 15032667748 kB VmStk: 132 kB VmExe: 15352 kB VmLib: 9024 kB VmPTE: 2372 kB VmSwap: 0 kB HugetlbPages: 0 kB CoreDumping: 0 THP_enabled: 1 Threads: 2 SigQ: 0/7729 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000001000 SigCgt: 00000001800046e7 CapInh: 0000000000000000 CapPrm: 0000003fffffffff CapEff: 0000003fffffffff CapBnd: 0000003fffffffff CapAmb: 0000000000000000 NoNewPrivs: 0 Seccomp: 0 Speculation_Store_Bypass: vulnerable Cpus_allowed: 3 Cpus_allowed_list: 0-1 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 427 nonvoluntary_ctxt_switches: 31 ```
cat /proc/$(pidof srs)/maps ``` 7fff7000-8fff7000 rwxp 00000000 00:00 0 8fff7000-2008fff7000 ---p 00000000 00:00 0 2008fff7000-10007fff8000 rwxp 00000000 00:00 0 564b3cbcc000-564b3ddb2000 r-xp 00000000 fc:01 1457467 /root/git/srs/trunk/objs/srs 564b3ddb2000-564b3de12000 r-xp 011e5000 fc:01 1457467 /root/git/srs/trunk/objs/srs 564b3de12000-564b3dea9000 rwxp 01245000 fc:01 1457467 /root/git/srs/trunk/objs/srs 564b3dea9000-564b3e8c1000 rwxp 00000000 00:00 0 600000000000-602000000000 ---p 00000000 00:00 0 602000000000-602000280000 rwxp 00000000 00:00 0 602000280000-602e00000000 ---p 00000000 00:00 0 602e00000000-602e00010000 rwxp 00000000 00:00 0 602e00010000-603000000000 ---p 00000000 00:00 0 603000000000-6030006d0000 rwxp 00000000 00:00 0 6030006d0000-603e00000000 ---p 00000000 00:00 0 603e00000000-603e00010000 rwxp 00000000 00:00 0 603e00010000-604000000000 ---p 00000000 00:00 0 604000000000-604000de0000 rwxp 00000000 00:00 0 604000de0000-604e00000000 ---p 00000000 00:00 0 604e00000000-604e00010000 rwxp 00000000 00:00 0 604e00010000-605000000000 ---p 00000000 00:00 0 605000000000-6050007e0000 rwxp 00000000 00:00 0 6050007e0000-605e00000000 ---p 00000000 00:00 0 605e00000000-605e00010000 rwxp 00000000 00:00 0 605e00010000-606000000000 ---p 00000000 00:00 0 606000000000-6060009d0000 rwxp 00000000 00:00 0 6060009d0000-606e00000000 ---p 00000000 00:00 0 606e00000000-606e00010000 rwxp 00000000 00:00 0 606e00010000-607000000000 ---p 00000000 00:00 0 607000000000-607000360000 rwxp 00000000 00:00 0 607000360000-607e00000000 ---p 00000000 00:00 0 607e00000000-607e00010000 rwxp 00000000 00:00 0 607e00010000-608000000000 ---p 00000000 00:00 0 608000000000-608000440000 rwxp 00000000 00:00 0 608000440000-608e00000000 ---p 00000000 00:00 0 608e00000000-608e00010000 rwxp 00000000 00:00 0 608e00010000-609000000000 ---p 00000000 00:00 0 609000000000-609000010000 rwxp 00000000 00:00 0 609000010000-609e00000000 ---p 00000000 00:00 0 609e00000000-609e00010000 rwxp 00000000 00:00 0 609e00010000-60b000000000 ---p 00000000 00:00 0 60b000000000-60b000290000 rwxp 00000000 00:00 0 60b000290000-60be00000000 ---p 00000000 00:00 0 60be00000000-60be00010000 rwxp 00000000 00:00 0 60be00010000-60c000000000 ---p 00000000 00:00 0 60c000000000-60c000650000 rwxp 00000000 00:00 0 60c000650000-60ce00000000 ---p 00000000 00:00 0 60ce00000000-60ce00010000 rwxp 00000000 00:00 0 60ce00010000-60d000000000 ---p 00000000 00:00 0 60d000000000-60d0001f0000 rwxp 00000000 00:00 0 60d0001f0000-60de00000000 ---p 00000000 00:00 0 60de00000000-60de00010000 rwxp 00000000 00:00 0 60de00010000-60e000000000 ---p 00000000 00:00 0 60e000000000-60e000320000 rwxp 00000000 00:00 0 60e000320000-60ee00000000 ---p 00000000 00:00 0 60ee00000000-60ee00010000 rwxp 00000000 00:00 0 60ee00010000-60f000000000 ---p 00000000 00:00 0 60f000000000-60f0001a0000 rwxp 00000000 00:00 0 60f0001a0000-60fe00000000 ---p 00000000 00:00 0 60fe00000000-60fe00010000 rwxp 00000000 00:00 0 60fe00010000-610000000000 ---p 00000000 00:00 0 610000000000-610001500000 rwxp 00000000 00:00 0 610001500000-610e00000000 ---p 00000000 00:00 0 610e00000000-610e00010000 rwxp 00000000 00:00 0 610e00010000-611000000000 ---p 00000000 00:00 0 611000000000-611001590000 rwxp 00000000 00:00 0 611001590000-611e00000000 ---p 00000000 00:00 0 611e00000000-611e00010000 rwxp 00000000 00:00 0 611e00010000-612000000000 ---p 00000000 00:00 0 612000000000-6120000b0000 rwxp 00000000 00:00 0 6120000b0000-612e00000000 ---p 00000000 00:00 0 612e00000000-612e00010000 rwxp 00000000 00:00 0 612e00010000-613000000000 ---p 00000000 00:00 0 613000000000-613000060000 rwxp 00000000 00:00 0 613000060000-613e00000000 ---p 00000000 00:00 0 613e00000000-613e00010000 rwxp 00000000 00:00 0 613e00010000-614000000000 ---p 00000000 00:00 0 614000000000-614000060000 rwxp 00000000 00:00 0 614000060000-614e00000000 ---p 00000000 00:00 0 614e00000000-614e00010000 rwxp 00000000 00:00 0 614e00010000-615000000000 ---p 00000000 00:00 0 615000000000-6150001d0000 rwxp 00000000 00:00 0 6150001d0000-615e00000000 ---p 00000000 00:00 0 615e00000000-615e00010000 rwxp 00000000 00:00 0 615e00010000-616000000000 ---p 00000000 00:00 0 616000000000-6160000b0000 rwxp 00000000 00:00 0 6160000b0000-616e00000000 ---p 00000000 00:00 0 616e00000000-616e00010000 rwxp 00000000 00:00 0 616e00010000-617000000000 ---p 00000000 00:00 0 617000000000-617000100000 rwxp 00000000 00:00 0 617000100000-617e00000000 ---p 00000000 00:00 0 617e00000000-617e00010000 rwxp 00000000 00:00 0 617e00010000-618000000000 ---p 00000000 00:00 0 618000000000-6180000a0000 rwxp 00000000 00:00 0 6180000a0000-618e00000000 ---p 00000000 00:00 0 618e00000000-618e00010000 rwxp 00000000 00:00 0 618e00010000-619000000000 ---p 00000000 00:00 0 619000000000-619000410000 rwxp 00000000 00:00 0 619000410000-619e00000000 ---p 00000000 00:00 0 619e00000000-619e00010000 rwxp 00000000 00:00 0 619e00010000-61a000000000 ---p 00000000 00:00 0 61a000000000-61a0001c0000 rwxp 00000000 00:00 0 61a0001c0000-61ae00000000 ---p 00000000 00:00 0 61ae00000000-61ae00010000 rwxp 00000000 00:00 0 61ae00010000-61b000000000 ---p 00000000 00:00 0 61b000000000-61b002330000 rwxp 00000000 00:00 0 61b002330000-61be00000000 ---p 00000000 00:00 0 61be00000000-61be00010000 rwxp 00000000 00:00 0 61be00010000-61c000000000 ---p 00000000 00:00 0 61c000000000-61c000200000 rwxp 00000000 00:00 0 61c000200000-61ce00000000 ---p 00000000 00:00 0 61ce00000000-61ce00010000 rwxp 00000000 00:00 0 61ce00010000-61d000000000 ---p 00000000 00:00 0 61d000000000-61d0003e0000 rwxp 00000000 00:00 0 61d0003e0000-61de00000000 ---p 00000000 00:00 0 61de00000000-61de00010000 rwxp 00000000 00:00 0 61de00010000-61e000000000 ---p 00000000 00:00 0 61e000000000-61e000300000 rwxp 00000000 00:00 0 61e000300000-61ee00000000 ---p 00000000 00:00 0 61ee00000000-61ee00010000 rwxp 00000000 00:00 0 61ee00010000-61f000000000 ---p 00000000 00:00 0 61f000000000-61f000220000 rwxp 00000000 00:00 0 61f000220000-61fe00000000 ---p 00000000 00:00 0 61fe00000000-61fe00010000 rwxp 00000000 00:00 0 61fe00010000-620000000000 ---p 00000000 00:00 0 620000000000-6200000f0000 rwxp 00000000 00:00 0 6200000f0000-620e00000000 ---p 00000000 00:00 0 620e00000000-620e00010000 rwxp 00000000 00:00 0 620e00010000-621000000000 ---p 00000000 00:00 0 621000000000-621000190000 rwxp 00000000 00:00 0 621000190000-621e00000000 ---p 00000000 00:00 0 621e00000000-621e00010000 rwxp 00000000 00:00 0 621e00010000-622000000000 ---p 00000000 00:00 0 622000000000-6220000a0000 rwxp 00000000 00:00 0 6220000a0000-622e00000000 ---p 00000000 00:00 0 622e00000000-622e00010000 rwxp 00000000 00:00 0 622e00010000-623000000000 ---p 00000000 00:00 0 623000000000-6230084e0000 rwxp 00000000 00:00 0 6230084e0000-623e00000000 ---p 00000000 00:00 0 623e00000000-623e00010000 rwxp 00000000 00:00 0 623e00010000-624000000000 ---p 00000000 00:00 0 624000000000-6240009b0000 rwxp 00000000 00:00 0 6240009b0000-624e00000000 ---p 00000000 00:00 0 624e00000000-624e00010000 rwxp 00000000 00:00 0 624e00010000-625000000000 ---p 00000000 00:00 0 625000000000-6250000b0000 rwxp 00000000 00:00 0 6250000b0000-625e00000000 ---p 00000000 00:00 0 625e00000000-625e00010000 rwxp 00000000 00:00 0 625e00010000-626000000000 ---p 00000000 00:00 0 626000000000-626000040000 rwxp 00000000 00:00 0 626000040000-626e00000000 ---p 00000000 00:00 0 626e00000000-626e00010000 rwxp 00000000 00:00 0 626e00010000-627000000000 ---p 00000000 00:00 0 627000000000-627000030000 rwxp 00000000 00:00 0 627000030000-627e00000000 ---p 00000000 00:00 0 627e00000000-627e00010000 rwxp 00000000 00:00 0 627e00010000-628000000000 ---p 00000000 00:00 0 628000000000-628000020000 rwxp 00000000 00:00 0 628000020000-628e00000000 ---p 00000000 00:00 0 628e00000000-628e00010000 rwxp 00000000 00:00 0 628e00010000-629000000000 ---p 00000000 00:00 0 629000000000-629000060000 rwxp 00000000 00:00 0 629000060000-629e00000000 ---p 00000000 00:00 0 629e00000000-629e00010000 rwxp 00000000 00:00 0 629e00010000-62a000000000 ---p 00000000 00:00 0 62a000000000-62a000070000 rwxp 00000000 00:00 0 62a000070000-62ae00000000 ---p 00000000 00:00 0 62ae00000000-62ae00010000 rwxp 00000000 00:00 0 62ae00010000-62b000000000 ---p 00000000 00:00 0 62b000000000-62b000020000 rwxp 00000000 00:00 0 62b000020000-62be00000000 ---p 00000000 00:00 0 62be00000000-62be00010000 rwxp 00000000 00:00 0 62be00010000-62c000000000 ---p 00000000 00:00 0 62c000000000-62c000010000 rwxp 00000000 00:00 0 62c000010000-62ce00000000 ---p 00000000 00:00 0 62ce00000000-62ce00010000 rwxp 00000000 00:00 0 62ce00010000-62d000000000 ---p 00000000 00:00 0 62d000000000-62d000030000 rwxp 00000000 00:00 0 62d000030000-62de00000000 ---p 00000000 00:00 0 62de00000000-62de00010000 rwxp 00000000 00:00 0 62de00010000-62f000000000 ---p 00000000 00:00 0 62f000000000-62f000040000 rwxp 00000000 00:00 0 62f000040000-62fe00000000 ---p 00000000 00:00 0 62fe00000000-62fe00010000 rwxp 00000000 00:00 0 62fe00010000-631000000000 ---p 00000000 00:00 0 631000000000-631000090000 rwxp 00000000 00:00 0 631000090000-631e00000000 ---p 00000000 00:00 0 631e00000000-631e00010000 rwxp 00000000 00:00 0 631e00010000-632000000000 ---p 00000000 00:00 0 632000000000-632000030000 rwxp 00000000 00:00 0 632000030000-632e00000000 ---p 00000000 00:00 0 632e00000000-632e00010000 rwxp 00000000 00:00 0 632e00010000-640000000000 ---p 00000000 00:00 0 640000000000-640000003000 rwxp 00000000 00:00 0 7f5a4c477000-7f5a4d9b3000 rwxp 00000000 00:00 0 7f5a4d9b3000-7f5a4d9b4000 ---p 00000000 00:00 0 7f5a4d9b4000-7f5a4e500000 rwxp 00000000 00:00 0 7f5a4e504000-7f5a4e700000 rwxp 00000000 00:00 0 7f5a4e708000-7f5a4ea00000 rwxp 00000000 00:00 0 7f5a4ea13000-7f5a50c38000 rwxp 00000000 00:00 0 7f5a50c38000-7f5a51418000 ---p 00000000 00:00 0 7f5a51418000-7f5a51776000 rwxp 00000000 00:00 0 7f5a51776000-7f5a5195e000 r-xp 00000000 fc:01 1195246 /usr/lib/x86_64-linux-gnu/libc-2.31.so 7f5a5195e000-7f5a51962000 r-xp 001e7000 fc:01 1195246 /usr/lib/x86_64-linux-gnu/libc-2.31.so 7f5a51962000-7f5a51964000 rwxp 001eb000 fc:01 1195246 /usr/lib/x86_64-linux-gnu/libc-2.31.so 7f5a51964000-7f5a5196a000 rwxp 00000000 00:00 0 7f5a5196a000-7f5a51983000 r-xp 00000000 fc:01 1182614 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 7f5a51983000-7f5a51984000 r-xp 00018000 fc:01 1182614 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 7f5a51984000-7f5a51985000 rwxp 00019000 fc:01 1182614 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 7f5a51985000-7f5a51ad2000 r-xp 00000000 fc:01 1195856 /usr/lib/x86_64-linux-gnu/libm-2.31.so 7f5a51ad2000-7f5a51ad3000 r-xp 0014c000 fc:01 1195856 /usr/lib/x86_64-linux-gnu/libm-2.31.so 7f5a51ad3000-7f5a51ad4000 rwxp 0014d000 fc:01 1195856 /usr/lib/x86_64-linux-gnu/libm-2.31.so 7f5a51ad4000-7f5a51ca3000 r-xp 00000000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7f5a51ca3000-7f5a51ca4000 ---p 001cf000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7f5a51ca4000-7f5a51caf000 r-xp 001cf000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7f5a51caf000-7f5a51cb2000 rwxp 001da000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7f5a51cb2000-7f5a51cb5000 rwxp 00000000 00:00 0 7f5a51cb5000-7f5a51cbd000 r-xp 00000000 fc:01 1195869 /usr/lib/x86_64-linux-gnu/librt-2.31.so 7f5a51cbd000-7f5a51cbe000 r-xp 00007000 fc:01 1195869 /usr/lib/x86_64-linux-gnu/librt-2.31.so 7f5a51cbe000-7f5a51cbf000 rwxp 00008000 fc:01 1195869 /usr/lib/x86_64-linux-gnu/librt-2.31.so 7f5a51cbf000-7f5a51cdc000 r-xp 00000000 fc:01 1195867 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so 7f5a51cdc000-7f5a51cdd000 r-xp 0001c000 fc:01 1195867 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so 7f5a51cdd000-7f5a51cde000 rwxp 0001d000 fc:01 1195867 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so 7f5a51cde000-7f5a51ce2000 rwxp 00000000 00:00 0 7f5a51ce2000-7f5a51ce6000 r-xp 00000000 fc:01 1195854 /usr/lib/x86_64-linux-gnu/libdl-2.31.so 7f5a51ce6000-7f5a51ce7000 r-xp 00003000 fc:01 1195854 /usr/lib/x86_64-linux-gnu/libdl-2.31.so 7f5a51ce7000-7f5a51ce8000 rwxp 00004000 fc:01 1195854 /usr/lib/x86_64-linux-gnu/libdl-2.31.so 7f5a51ce8000-7f5a51ceb000 rwxp 00000000 00:00 0 7f5a51ceb000-7f5a51cf5000 rwxp 00000000 00:00 0 7f5a51cf5000-7f5a51d21000 r-xp 00000000 fc:01 1195242 /usr/lib/x86_64-linux-gnu/ld-2.31.so 7f5a51d21000-7f5a51d22000 rwxp 00000000 00:00 0 7f5a51d22000-7f5a51d23000 r-xp 0002c000 fc:01 1195242 /usr/lib/x86_64-linux-gnu/ld-2.31.so 7f5a51d23000-7f5a51d24000 rwxp 0002d000 fc:01 1195242 /usr/lib/x86_64-linux-gnu/ld-2.31.so 7f5a51d24000-7f5a51d25000 rwxp 00000000 00:00 0 7ffd2f5a1000-7ffd2f5c2000 rwxp 00000000 00:00 0 [stack] 7ffd2f5e7000-7ffd2f5ea000 r--p 00000000 00:00 0 [vvar] 7ffd2f5ea000-7ffd2f5eb000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall] ```

The backtrace with gdb:

gdb --args ./objs/srs -c conf/console.conf ``` (gdb) bt #0 run_in_thread_pool () at ./src/main/srs_main_server.cpp:462 #1 0x00005555560d7372 in run_directly_or_daemon () at ./src/main/srs_main_server.cpp:414 #2 0x00005555560d4c1e in do_main (argc=3, argv=0x7fffffffe528, envp=0x7fffffffe548) at ./src/main/srs_main_server.cpp:242 #3 0x00005555560d4f2f in main (argc=3, argv=0x7fffffffe528, envp=0x7fffffffe548) at ./src/main/srs_main_server.cpp:253 (gdb) bt #0 SrsServerAdapter::run (this=0x602000001010, wg=0x7ffff43b2b50) at ./src/app/srs_app_server.cpp:1377 #1 0x0000555555f8e0e8 in SrsHybridServer::run (this=0x607000000f00) at ./src/app/srs_app_hybrid.cpp:222 #2 0x00005555560d7ea3 in run_hybrid_server () at ./src/main/srs_main_server.cpp:518 #3 0x0000555555f980ee in SrsThreadPool::start (arg=0x60c000000400) at ./src/app/srs_app_threads.cpp:793 #4 0x00007ffff7f9d609 in start_thread (arg=) at pthread_create.c:477 #5 0x00007ffff7b6b133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (gdb) bt #0 SrsServer::cycle (this=0x570fcd6e7c210700) at ./src/app/srs_app_server.cpp:863 #1 0x0000555555d54eee in SrsFastCoroutine::cycle (this=0x60e0000072e0) at ./src/app/srs_app_st.cpp:285 #2 0x0000555555d55030 in SrsFastCoroutine::pfn (arg=0x60e0000072e0) at ./src/app/srs_app_st.cpp:300 #3 0x00005555560d8fb2 in _st_thread_main () at sched.c:380 #4 0x00005555560d98d8 in st_thread_create (start=0x555555d5500c , arg=0x60e000737200, joinable=1698246705, stk_size=828911667) at sched.c:666 Backtrace stopped: previous frame inner to this frame (corrupt stack?) (gdb) bt #0 st_read (fd=0x7ffff34cc840, buf=0x7ffff34cc740, nbyte=140737275283360, timeout=15799937222417039645) at io.c:350 #1 0x0000555555c5f435 in SrsStSocket::read (this=0x607000004fc0, buf=0x7ffff470b800, size=131072, nread=0x7ffff34cc760) at ./src/protocol/srs_protocol_st.cpp:547 #2 0x0000555555cb2cfb in SrsTcpConnection::read (this=0x604000003f50, buf=0x7ffff470b800, size=131072, nread=0x7ffff34cc760) at ./src/app/srs_app_conn.cpp:568 #3 0x0000555555c0545c in SrsFastStream::grow (this=0x606000005480, reader=0x604000003f50, required_size=1) at ./src/protocol/srs_protocol_stream.cpp:161 #4 0x0000555555baaf0c in SrsProtocol::read_basic_header (this=0x611000010300, fmt=@0x7ffff34cc860: 0 '\000', cid=@0x7ffff34cc870: 0) at ./src/protocol/srs_protocol_rtmp_stack.cpp:891 #5 0x0000555555baa7b6 in SrsProtocol::recv_interlaced_message (this=0x611000010300, pmsg=0x7ffff34cc9a0) at ./src/protocol/srs_protocol_rtmp_stack.cpp:797 #6 0x0000555555ba64ea in SrsProtocol::recv_message (this=0x611000010300, pmsg=0x7ffff34cca70) at ./src/protocol/srs_protocol_rtmp_stack.cpp:339 #7 0x0000555555be0857 in SrsProtocol::expect_message (this=0x611000010300, pmsg=0x7ffff34ccbf0, ppacket=0x7ffff34ccc10) at ./src/protocol/srs_protocol_rtmp_stack.hpp:307 #8 0x0000555555bdaf94 in SrsRtmpServer::expect_message (this=0x603000008680, pmsg=0x7ffff34ccbf0, ppacket=0x7ffff34ccc10) at ./src/protocol/srs_protocol_rtmp_stack.hpp:773 #9 0x0000555555bbb5de in SrsRtmpServer::connect_app (this=0x603000008680, req=0x615000013900) at ./src/protocol/srs_protocol_rtmp_stack.cpp:2247 #10 0x0000555555ccbabf in SrsRtmpConn::do_cycle (this=0x61200005d340) at ./src/app/srs_app_rtmp_conn.cpp:217 #11 0x0000555555ce25a6 in SrsRtmpConn::cycle (this=0x61200005d340) at ./src/app/srs_app_rtmp_conn.cpp:1609 #12 0x0000555555d54eee in SrsFastCoroutine::cycle (this=0x60e0000080e0) at ./src/app/srs_app_st.cpp:285 #13 0x0000555555d55030 in SrsFastCoroutine::pfn (arg=0x60e0000080e0) at ./src/app/srs_app_st.cpp:300 #14 0x00005555560d8fb2 in _st_thread_main () at sched.c:380 #15 0x00005555560d98d8 in st_thread_create (start=0x604000003fd0, arg=0x7ffff4c25ee0, joinable=1460653422, stk_size=2082539264) at sched.c:666 #16 0x0000555555c59d47 in srs_context_set_cid_of (trd=0x7ffff4c26648, v=...) at ./src/protocol/srs_protocol_log.cpp:91 #17 0x0000555555a579e1 in _SrsContextId::~_SrsContextId (this=0x7ffff4c26298, __in_chrg=) at ./src/core/srs_core.cpp:24 #18 0x0000555555c59eeb in impl_SrsContextRestore::~impl_SrsContextRestore (this=0x7ffff4c26290, __in_chrg=) at ./src/protocol/srs_protocol_log.cpp:101 #19 0x0000555555ca9532 in SrsServer::do_on_tcp_client (this=0x61100000ff40, listener=0x6040000020d0, stfd=@0x7ffff4c263b0: 0xffffe984c94) at ./src/app/srs_app_server.cpp:1149 #20 0x00007ffff4791f20 in ?? () #21 0x00007ffff4c26648 in ?? () #22 0x00007ffff4c263c0 in ?? () #23 0x0000000100000001 in ?? () #24 0x570fcd6e7c210700 in ?? () #25 0x00007ffff4c263d0 in ?? () #26 0x00005555560db1a5 in st_netfd_poll (fd=0x0, how=0, timeout=0) at io.c:249 ```
winlinvip commented 2 months ago

Note that I have tried the following actions:

So it is caused by libasan, you can disable asan by --sanitizer=off to bypass this issue.

After disabled asan:

ps -u -p $(pidof srs) ``` USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 3157553 0.4 1.3 40824 27312 pts/0 S+ 10:49 0:01 /root/git/srs/trunk/objs/srs -c conf/console.conf ```
top -p $(pidof srs) ``` top - 10:54:14 up 144 days, 23:52, 3 users, load average: 1.40, 0.80, 0.66 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 10.0 us, 3.3 sy, 0.0 ni, 86.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 1987.4 total, 234.4 free, 1236.1 used, 516.9 buff/cache MiB Swap: 1025.0 total, 621.2 free, 403.8 used. 573.2 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3157553 root 20 0 40824 27312 13364 S 0.0 1.3 0:01.26 srs ```
cat /proc/$(pidof srs)/status ``` Name: srs Umask: 0022 State: S (sleeping) Tgid: 3157553 Ngid: 0 Pid: 3157553 PPid: 3157549 TracerPid: 3157549 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 64 Groups: 0 NStgid: 3157553 NSpid: 3157553 NSpgid: 3157553 NSsid: 3113227 VmPeak: 40848 kB VmSize: 40824 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 27312 kB VmRSS: 27312 kB RssAnon: 13948 kB RssFile: 13364 kB RssShmem: 0 kB VmData: 23184 kB VmStk: 132 kB VmExe: 9464 kB VmLib: 8028 kB VmPTE: 100 kB VmSwap: 0 kB HugetlbPages: 0 kB CoreDumping: 0 THP_enabled: 1 Threads: 1 SigQ: 0/7729 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000001000 SigCgt: 0000000180004227 CapInh: 0000000000000000 CapPrm: 0000003fffffffff CapEff: 0000003fffffffff CapBnd: 0000003fffffffff CapAmb: 0000000000000000 NoNewPrivs: 0 Seccomp: 0 Speculation_Store_Bypass: vulnerable Cpus_allowed: 3 Cpus_allowed_list: 0-1 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 33266 nonvoluntary_ctxt_switches: 78 ```
cat /proc/$(pidof srs)/maps ``` 555555554000-55555608b000 r-xp 00000000 fc:01 1457467 /root/git/srs/trunk/objs/srs 55555608b000-5555560ea000 r-xp 00b36000 fc:01 1457467 /root/git/srs/trunk/objs/srs 5555560ea000-5555560f5000 rwxp 00b95000 fc:01 1457467 /root/git/srs/trunk/objs/srs 5555560f5000-5555563c2000 rwxp 00000000 00:00 0 [heap] 7ffff66a4000-7ffff7a58000 rwxp 00000000 00:00 0 7ffff7a58000-7ffff7c40000 r-xp 00000000 fc:01 1195246 /usr/lib/x86_64-linux-gnu/libc-2.31.so 7ffff7c40000-7ffff7c44000 r-xp 001e7000 fc:01 1195246 /usr/lib/x86_64-linux-gnu/libc-2.31.so 7ffff7c44000-7ffff7c46000 rwxp 001eb000 fc:01 1195246 /usr/lib/x86_64-linux-gnu/libc-2.31.so 7ffff7c46000-7ffff7c4a000 rwxp 00000000 00:00 0 7ffff7c4a000-7ffff7c63000 r-xp 00000000 fc:01 1182614 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 7ffff7c63000-7ffff7c64000 r-xp 00018000 fc:01 1182614 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 7ffff7c64000-7ffff7c65000 rwxp 00019000 fc:01 1182614 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 7ffff7c65000-7ffff7db2000 r-xp 00000000 fc:01 1195856 /usr/lib/x86_64-linux-gnu/libm-2.31.so 7ffff7db2000-7ffff7db3000 r-xp 0014c000 fc:01 1195856 /usr/lib/x86_64-linux-gnu/libm-2.31.so 7ffff7db3000-7ffff7db4000 rwxp 0014d000 fc:01 1195856 /usr/lib/x86_64-linux-gnu/libm-2.31.so 7ffff7db4000-7ffff7f83000 r-xp 00000000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7ffff7f83000-7ffff7f84000 ---p 001cf000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7ffff7f84000-7ffff7f8f000 r-xp 001cf000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7ffff7f8f000-7ffff7f92000 rwxp 001da000 fc:01 1182754 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 7ffff7f92000-7ffff7f95000 rwxp 00000000 00:00 0 7ffff7f95000-7ffff7fb2000 r-xp 00000000 fc:01 1195867 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so 7ffff7fb2000-7ffff7fb3000 r-xp 0001c000 fc:01 1195867 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so 7ffff7fb3000-7ffff7fb4000 rwxp 0001d000 fc:01 1195867 /usr/lib/x86_64-linux-gnu/libpthread-2.31.so 7ffff7fb4000-7ffff7fb8000 rwxp 00000000 00:00 0 7ffff7fb8000-7ffff7fbc000 r-xp 00000000 fc:01 1195854 /usr/lib/x86_64-linux-gnu/libdl-2.31.so 7ffff7fbc000-7ffff7fbd000 r-xp 00003000 fc:01 1195854 /usr/lib/x86_64-linux-gnu/libdl-2.31.so 7ffff7fbd000-7ffff7fbe000 rwxp 00004000 fc:01 1195854 /usr/lib/x86_64-linux-gnu/libdl-2.31.so 7ffff7fbe000-7ffff7fc0000 rwxp 00000000 00:00 0 7ffff7fcb000-7ffff7fce000 r--p 00000000 00:00 0 [vvar] 7ffff7fce000-7ffff7fcf000 r-xp 00000000 00:00 0 [vdso] 7ffff7fcf000-7ffff7ffb000 r-xp 00000000 fc:01 1195242 /usr/lib/x86_64-linux-gnu/ld-2.31.so 7ffff7ffc000-7ffff7ffd000 r-xp 0002c000 fc:01 1195242 /usr/lib/x86_64-linux-gnu/ld-2.31.so 7ffff7ffd000-7ffff7ffe000 rwxp 0002d000 fc:01 1195242 /usr/lib/x86_64-linux-gnu/ld-2.31.so 7ffff7ffe000-7ffff7fff000 rwxp 00000000 00:00 0 7ffffffde000-7ffffffff000 rwxp 00000000 00:00 0 [stack] ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall] ```