[Bug]: Server Crash - Large Episode Count (Still an issue)

CorneliousJD commented 15 hours ago

What happened?

This was apparently fixed a long time ago but I'm still seeing the same issue.

Below logs are all I get then it just hard-crashes.

What did you expect to happen?

That it refreshes and downloads

Steps to reproduce the issue

Download "Stuff you should know" podcast.

Audiobookshelf version

2.16.2

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

Other (list in "Additional Notes" box)

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

Socket failed to connect in web UI
nothing in docker logs whatsoever, but here they are anyways.

[2024-11-12 19:47:06.719] INFO: === Starting Server ===
[2024-11-12 19:47:06.721] INFO: [Server] Init v2.16.2
[2024-11-12 19:47:06.722] INFO: [Server] Node.js Version: v20.18.0
[2024-11-12 19:47:06.722] INFO: [Server] Platform: linux
[2024-11-12 19:47:06.723] INFO: [Server] Arch: x64
[2024-11-12 19:47:06.730] INFO: [Database] Initializing db at "/config/absdatabase.sqlite"
[2024-11-12 19:47:06.779] INFO: [Database] Loading extension /usr/local/lib/nusqlite3/libnusqlite3.so
[2024-11-12 19:47:06.781] INFO: [Database] Successfully loaded extension /usr/local/lib/nusqlite3/libnusqlite3.so
[2024-11-12 19:47:06.781] INFO: [Database] Db supports unaccent and unicode foldings
[2024-11-12 19:47:06.782] INFO: [Database] Db connection was successful
[2024-11-12 19:47:06.791] INFO: [MigrationManager] Database is already up to date.
[2024-11-12 19:47:07.024] INFO: [Database] Db initialized with models: user, library, libraryFolder, book, podcast, podcastEpisode, libraryItem, mediaProgress, series, bookSeries, author, bookAuthor, collection, collectionBook, playlist, playlistMediaItem, device, playbackSession, feed, feedEpisode, setting, customMetadataProvider, mediaItemShare
[2024-11-12 19:47:07.066] INFO: [LogManager] Removed daily log: 2024-03-24.txt
[2024-11-12 19:47:07.067] INFO: [LogManager] Init current daily log filename: 2024-11-12.txt
[2024-11-12 19:47:07.114] INFO: [BackupManager] 7 Backups Found
[2024-11-12 19:47:07.180] INFO: [Watcher] Initializing watcher for "Audio Books".
[2024-11-12 19:47:07.184] INFO: [Watcher] Initializing watcher for "Podcasts".
[2024-11-12 19:47:07.203] INFO: Listening on port :80
[2024-11-12 19:47:07.228] INFO: [Watcher] "Audio Books" Ready
[2024-11-12 19:47:07.794] INFO: [Watcher] "Podcasts" Ready
[2024-11-12 20:00:02.553] INFO: [PodcastManager] runEpisodeCheck: "Stuff You Should Know" | Last check: Thu Jul 20 2023 04:00:07 GMT-0400 (Eastern Daylight Time) | Latest episode pubDate: Wed Jul 19 2023 05:00:00 GMT-0400 (Eastern Daylight Time)
[2024-11-12 20:00:05.794] INFO: [PodcastManager] Found 3 new episodes for podcast "Stuff You Should Know" - starting download



### Additional Notes

unRAID

nichwall commented 15 hours ago

Is it crashing while downloading or after the podcast episodes have been downloaded? How many podcast episodes are successfully downloaded?

(Edited to clarify the difference between podcast and episode)

CorneliousJD commented 15 hours ago

Is it crashing while downloading or after the podcast has been downloaded? How many podcasts are successfully downloaded?

it crashes when trying to search for, then download new files. it will search them just fine but it immediately crashes when trying to download.

it's been an issue for a long time and i thought it was related to the "issue w/ podcasts w/ lots of episodes" thing but that didn't seem to fix it.

about 1300 of over 2k episodes have successfully downloaded from many many months ago, but nothing new for a long time now.

would it be worth it to just blow my whole instance away and start over? I can easily re-add and try re-downloading if i have to.

nichwall commented 14 hours ago

Thanks, the crash during download is likely due to running out of memory and should be resolved by https://github.com/advplyr/audiobookshelf/pull/3604 which is available in the edge image and will be in the next release.

You can take a look at crash_logs in your /metadata/logs folder to get more crash log information. You can also change the log level to Debug in the server settings to get more information from the logs.

We are currently investigating the high memory usage for large podcasts, but Out of Memory reports all kind of get clumped together so it is difficult to get exact reproducible steps.

Does it crash when you are just browsing the podcast library (no downloads in progress)?
Do you have the auto-download on for the podcast?
Does it crash even if you are only trying to download a single episode?

Starting over won't fix it if you are running out of memory for the container, since it is just that the podcast is getting too large. As a workaround, you can increase the maximum memory for the server by adding the environment variable NODE_OPTIONS with a value of --max-old-space-size=8192, so something like NODE_OPTIONS=--max-old-space-size=8192, depending on how you set that in unraid. https://nodejs.org/api/cli.html#--max-old-space-sizesize-in-megabytes

advplyr / audiobookshelf