Closed verybadsoldier closed 7 months ago
C9Glax
commented
1 year ago Can confirm same issue, maybe related: I had a lot of open playerse, because they never get deleted.
For a temporary workaround i put cpu: 1 in the docker-compose.yaml put an end to my server literally dying.
kagemomiji
commented
1 year ago @verybadsoldier
It appears from your logs that the URL specified for the podcast (https://www.podcast.de/episode/595537228/ausgabe-fuenfzig) is incorrect, leading to a failure in retrieving the episodes. To confirm if this is the cause of the bug, I would like to either remove the affected podcast or update it with the correct URL.
I believe the correct URL should be https://lanz-precht.podigee.io/feed/mp3.
Could you please remove the podcast entry or update it with the correct URL? This will help in verifying if the function for updating podcasts causing the issue.
Thank you for your cooperation. I look forward to your response.
verybadsoldier
commented
1 year ago Ok, thank you. I removed it. I also noticed that my podcast folder was not writable. So podcasts could not be downloaded anyway. I fixed that now.
But, this CPU problem did not happen again since I reported it. But the podcast error appears regularly in the log (once a day?). So, according to me feeling, those are not related. But I will observe.
khers
commented
1 year ago I have seen the same behavior and do not have any exceptions in the log that look promising (only a podcast whose host was not alive). The settings page claimed it was still scanning media folders.
The-Exterminator
commented
1 year ago Raspberry do the same sometime, and restart the Raspberry.
fault1er1
commented
1 year ago Have the same issue on Windows Server running the .war directly from the cmd.
The-Exterminator
commented
1 year ago The Old airsonic and airsonic-advance are more stabil end this version. 😢 And i drive Raspberry 4 8gb
kagemomiji
commented
1 year ago I'm unable to reproduce this in my environment. Could you provide me the results of the top -H command on the Linux OS when the CPU usage reaches 100%? If it's running in Docker, the results from the machine hosting Docker would be fine.
The-Exterminator
commented
1 year ago I don't drive it with docker, But .war
kagemomiji
commented
1 year ago @The-Exterminator It would be helpful if you could provide the result of the previous command when the issue occurs, even if you are not hosting it with Docker.
The-Exterminator
commented
1 year ago @kagemomiji write the commando i Will try it when i come home
kagemomiji
commented
1 year ago @The-Exterminator
thank you! I mentioned the command here.
top -H
I'm unable to reproduce this in my environment. Could you provide me the results of the
top -Hcommand on the Linux OS when the CPU usage reaches 100%? If it's running in Docker, the results from the machine hosting Docker would be fine.
The-Exterminator
commented
1 year ago @kagemomiji Sorry, But what Will you have ???
Usage: top -hv | -bcEeHiOSs1 -d secs -n max -u|U user -p pid(s) -o field -w [cols]
kagemomiji
commented
1 year ago @The-Exterminator
It's -H, not -h
I want to identify which thread is consuming the CPU resources, so I need the results from the top -H command.
The-Exterminator
commented
1 year ago @kagemomiji Right now it looks like this, but it restarted last night again when airsonic started scanning and didn't finish. Can I make it save it in a log so if it crashes again I can send it to you. ????
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2442 pi 20 0 384852 36696 26848 S 5,7 0,5 15:00.82 lxterminal
1214814 tomcat 20 0 6088448 1,2g 50856 D 21,0 16,3 0:15.95 MediaLibrarySca
1214813 tomcat 20 0 6088448 1,2g 50856 D 18,4 16,3 0:15.40 MediaLibrarySca
1214815 tomcat 20 0 6088448 1,2g 50856 S 17,8 16,3 0:14.97 MediaLibrarySca
1214816 tomcat 20 0 6088448 1,2g 50856 D 17,8 16,3 0:15.43 MediaLibrarySca
1214800 tomcat 20 0 6088448 1,2g 50856 D 17,5 16,3 0:15.16 MediaLibrarySca
684 tomcat 20 0 6088448 1,2g 50856 S 16,8 16,3 0:39.80 G1 Conc#0
7771 mysql 20 0 2443656 359616 22184 R 16,2 4,5 1:15.25 mariadbd
456264 mysql 20 0 2443656 359616 22184 S 14,6 4,5 0:36.42 mariadbd
8001 mysql 20 0 2443656 359616 22184 S 10,0 4,5 6:11.95 mariadbd
699 tomcat 20 0 6088448 1,2g 50856 S 8,4 16,3 8:05.61 C2 CompilerThre
451223 mysql 20 0 2443656 359616 22184 S 7,1 4,5 2:18.34 mariadbd
1225 pi 20 0 603592 80996 57632 S 6,1 1,0 3:42.71 mutter
937 root 20 0 99004 16176 7920 S 4,5 0,2 35:59.70 python3
254 root -2 0 0 0 0 S 3,6 0,0 1:18.89 v3d_bin
255 root -2 0 0 0 0 S 3,6 0,0 1:20.62 v3d_render
784 tomcat 20 0 6088448 1,2g 50856 S 1,9 16,3 0:18.12 GC Thread#1
785 tomcat 20 0 6088448 1,2g 50856 S 1,6 16,3 0:17.90 GC Thread#2
1214787 mysql 20 0 2443656 359616 22184 S 1,6 4,5 0:00.77 mariadbd
55 root 20 0 0 0 0 S 1,3 0,0 0:27.21 kswapd0
682 tomcat 20 0 6088448 1,2g 50856 S 1,3 16,3 0:18.22 GC Thread#0
923 tomcat 20 0 6088448 1,2g 50856 S 1,3 16,3 0:18.03 GC Thread#3
1214785 mysql 20 0 2443656 359616 22184 R 1,3 4,5 0:00.69 mariadbd
1214835 mysql 20 0 2443656 359616 22184 S 1,3 4,5 0:00.72 mariadbd
1214836 mysql 20 0 2443656 359616 22184 S 1,3 4,5 0:00.64 mariadbd
445841 root 0 -20 0 0 0 I 1,3 0,0 0:32.92 kworker/0:1H-mmc_complete
1210845 root 20 0 10492 3780 2780 R 1,3 0,0 0:09.22 top
15 root 20 0 0 0 0 I 1,0 0,0 1:24.11 rcu_preempt
146 root 20 0 155700 98820 97608 S 1,0 1,2 3:19.97 systemd-journal
1214792 tomcat 20 0 6088448 1,2g 50856 S 1,0 16,3 0:00.54 Ehcache [_defau
687 mysql 20 0 2443656 359616 22184 S 1,0 4,5 0:26.71 mariadbd
7876 mysql 20 0 2443656 359616 22184 S 1,0 4,5 3:52.41 mariadbd
1212790 mysql 20 0 2443656 359616 22184 S 1,0 4,5 0:00.77 mariadbd
989 root 20 0 99004 16176 7920 S 1,0 0,2 4:25.53 python3
1214837 mysql 20 0 2443656 359616 22184 S 0,6 4,5 0:00.59 mariadbd
2651 root 20 0 4968 464 400 S 0,6 0,0 0:26.20 tail
1186787 root 20 0 0 0 0 I 0,6 0,0 0:00.44 kworker/u8:1-events_unbound
1208709 pi 20 0 93972 4972 4056 S 0,6 0,1 0:01.07 barrierc
25 root 20 0 0 0 0 S 0,3 0,0 0:07.88 ksoftirqd/2
491 root 20 0 220884 4428 2904 S 0,3 0,1 0:38.85 in:imuxsock
493 root 20 0 220884 4428 2904 S 0,3 0,1 0:47.40 rs:main Q:Reg
683 tomcat 20 0 6088448 1,2g 50856 S 0,3 16,3 0:00.22 G1 Main Marker
693 tomcat 20 0 6088448 1,2g 50856 S 0,3 16,3 0:05.34 VM Thread
1214788 mysql 20 0 2443656 359616 22184 S 0,3 4,5 0:00.63 mariadbd
1214823 mysql 20 0 2443656 359616 22184 R 0,3 4,5 0:00.56 mariadbd
1233 pi 20 0 10460 4948 4184 S 0,3 0,1 0:05.53 xscreensaver
458054 root 0 -20 0 0 0 I 0,3 0,0 0:06.07 kworker/2:0H-kblockd
1204554 root 20 0 0 0 0 I 0,3 0,0 0:00.08 kworker/1:2-events_power_efficientI had hit this problem myself several weeks ago and I updated to https://github.com/kagemomiji/airsonic-advanced/commit/8ccfc0d835fc99c6ac4430769c694fa6eb0b67ca (which was newest at the time) and have not seen it since, I suspect it has been fixed already.
verybadsoldier
commented
1 year ago Thanks everyone for looking into this! <3
kagemomiji
commented
1 year ago @The-Exterminator
Capturing data at the exact moment of a crash can be challenging, but you can save the results of the top -H command every second using the following command. If you know the exact time of the crash, you can cross-reference it with the data from the top.log produced by this command.
watch -n 1 "top -Hbo +%CPU | head -n 32 | tee -a top.log"
When you want to stop capturing logs, please press Ctrl + C in the terminal where you input the command.
Also, as @khers mentioned in their comment, the issue might have been resolved in the edge version. Could you please let me know which version you are using?
The-Exterminator
commented
1 year ago @kagemomiji I Will try that when i come Home again. The version i have, is the newest, and the version before, do the same.
The-Exterminator
commented
1 year ago Hello @kagemomiji It seems to be running now. I have replaced my usb key with an sd card and seems to run without rebooting. But airsonic and the old airsonic-advance have no problems running them with the usb key. So i think the bug is still there in this version you have.
kagemomiji
commented
7 months ago @verybadsoldier @The-Exterminator @fault1er1
I have been unable to resolve this for a long time, but the recent change (#337) includes an adjustment to set the scan thread's priority to the lowest.
I believe this may have solved the issue of the CPU being utilized at 100% . Could someone please verify this with the latest edge release?
kagemomiji
commented
7 months ago In my environment, by lowering the priority of threads, I have confirmed that the average CPU usage during MediaLibrary Scan has decreased from 90% to 50%. Additionally, the implementation of a fail-safe mechanism through timeouts has proven to be effective. If there are any remaining issues, please feel free to reopen this issue.
The-Exterminator
commented
7 months ago @kagemomiji i Can try it, on monday, and i Will write back 👍
What happened?
After running Airsonic for like 1 week it suddenly got to a state to permanently hog one CPU keeping it at 100%:
Happened at night and was not in use at that moment, as far as I know. I observed this behavior one time before.
It is still working fine and usable. After restarting the container, everything is ok again.
I appended some logs, but probably hard to say what the problem is?
Any other information I should provide or anything I should test or do to help out analyzing this?
Steps to reproduce
Version
11.1.2 (Stable)
Version Detail
No response
Operating System
Ubuntu 22.04.2 LTS VM
Java Version
ghcr.io/kagemomiji/airsonic-advanced:11.1.2
Database
Other
DB Detail
whatever is on the docker imagae
Configuration paramter
ker USER id: 0 Docker PUID env: 997 Docker USER group: 0 Docker PGID env: 996 Process will run as: User: 997 Group: 996 openjdk 17.0.5 2022-10-18 OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8) OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (build 17.0.5+8, mixed mode, sharing) JAVA_OPTS=-Xmx768m -Dserver.use-forward-headers=true -Dserver.forward-headers-strategy=native ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.13 Release-Date: 2022-01-05 Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin CONTEXTPATH=/ /\ () () / \ _ / /\ \ | | '|/ |/ | ' | |/ | / __ | | | __ \ () | | | | | ( // __|| |/\/|| ||_|___|
2023-05-24 21:31:35.307 INFO --- org.airsonic.player.Application : Starting Application using Java 17.0.5 on 873dc8c07a48 with PID 1 (/app/WEB-INF/classes started by au in /var) 2023-05-24 21:31:35.309 INFO --- org.airsonic.player.Application : No active profile set, falling back to 1 default profile: "default"
Proxy Server
No response
client detail
No response
language
None
Relevant log output