coturn / coturn

coturn TURN server project
Other
11.42k stars 2.03k forks source link

Segfault when accessing logs #1238

Open AustinLangston opened 1 year ago

AustinLangston commented 1 year ago

coTURN begins its scheduled restart and then can no longer start, it's failing with the error below. This is seen on 4.5.1 on Buster as well as 4.5.2 on Bullseye. We've seen this occur after being in service for as little as 2 weeks but have also seen the servers run for 5+ months without issue.

Apr 19 23:49:55 ip-10-170-9-85 turnserver[2142347]: 95282: : session 000000000000006527: reaERROR: Cannot open log file for writing: /var/log/coturn/turn_2023-04-19.log Apr 19 23:49:55 ip-10-170-9-85 kernel: [2524204.193537] turnserver[2142348]: segfault at c0 ip 00007f87cfa49cb0 sp 00007f87ce9d0ae0 error 4 in libc-2.31.so[7f87cfa05000+15a000] Apr 19 23:49:55 ip-10-170-9-85 kernel: [2524204.204515] Code: 48 81 ec 28 05 00 00 89 8d 38 fb ff ff 64 48 8b 04 25 28 00 00 00 48 89 45 c8 48 8b 05 c9 81 16 00 64 8b 00 89 85 24 fb ff ff <8b> 87 c0 00 00 00 85 c0 0f 85 f2 01 00 00 c7 87 c0 00 00 00 ff ff Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: coturn.service: Main process exited, code=killed, status=11/SEGV Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: coturn.service: Failed with result 'signal'. Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: coturn.service: Consumed 19min 58.785s CPU time. Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: coturn.service: Scheduled restart job, restart counter is at 1. Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: Stopped coTURN STUN/TURN Server. Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: coturn.service: Consumed 19min 58.785s CPU time. Apr 19 23:49:55 ip-10-170-9-85 systemd[1]: Starting coTURN STUN/TURN Server...

Permissions are the same, Disk isn't close to being full, and we have no way of reliably reproducing the issue.

eakraly commented 1 year ago

Hi @AustinLangston

4.5.2 is an old version and I recommend upgrading to latest (4.6.2 at the moment) - it contains multiple memory access fixes. Your sample makes me think it is one of them.