advplyr / audiobookshelf

Self-hosted audiobook and podcast server
https://audiobookshelf.org
GNU General Public License v3.0
6.23k stars 436 forks source link

[Enhancement]: Warning: connect.session() MemoryStore is not designed for a production environment #2538

Closed nichwall closed 2 weeks ago

nichwall commented 7 months ago

Describe the feature/enhancement

There have been a few reports in recent weeks around express-rate-limit using memory-store. This issue is to provide context.

MemoryStore is not designed for a production environment, as it will leak memory, and will not scale past a single process.

This warning does not matter for ABS, but has been causing some confusion for users. https://express-rate-limit.mintlify.app/reference/stores

The MemoryStore is only used to provide rate limiting and does not end to be very robust since it is designed for a single server and rebooting does not affect the rate limiting feature for ABS (ABS only tracks for the past 60 seconds).

Potential changes:

  1. Suppress the warning so it doesn't show up at all (don't like that idea)
  2. Use a different storage, such as rate-limit-memcached. That seems to be the lightest replacement but adds a dependency
zzyzx-dc commented 7 months ago

Thank you for posting this. I am troubleshooting why Audiobookshelf crashes every week or two and I saw this error message. Good to know it is not related to my issue.

nichwall commented 7 months ago

Thank you for posting this. I am troubleshooting why Audiobookshelf crashes every week or two and I saw this error message. Good to know it is not related to my issue.

Glad it helped!

If you can provide logs around the crash, that will help narrow it down. If you haven't already done so, change your server log level to Debug. You may also need to look at the Docker or systemctl logs (depending on how you are running the server) because those logs can catch more information than ABS itself does for crashes.

Please post those logs/description in a new issue or on Discord so this issue doesn't get a ton of unrelated comments. :)

Eschguy commented 5 months ago

Having this issue as well. Based on info logs, mine seems to crash after updating podcasts. I've set the logger to debug and will post results when it crashes next.

INFO logs from crash/startup

{"timestamp":"2024-04-01 15:00:03.285","source":"LibraryItem.js:389","message":"[LibraryItem] Library item \"0246f2a8-adfd-4912-ac7e-8e62049b29b1\" updated","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:03.285","source":"PodcastManager.js:239","message":"[PodcastManager] runEpisodeCheck: \"Life Kit\" | Last check: Mon Apr 01 2024 14:00:05 GMT-0500 (Central Daylight Time) | Latest episode pubDate: Mon Apr 01 2024 02:00:59 GMT-0500 (Central Daylight Time)","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:03.711","source":"LibraryItem.js:389","message":"[LibraryItem] Library item \"1ca69716-161f-4285-aaca-86e1716e4f22\" updated","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:03.711","source":"PodcastManager.js:239","message":"[PodcastManager] runEpisodeCheck: \"Stuff You Missed in History Class\" | Last check: Mon Apr 01 2024 14:00:06 GMT-0500 (Central Daylight Time) | Latest episode pubDate: Mon Apr 01 2024 08:00:00 GMT-0500 (Central Daylight Time)","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:05.426","source":"LibraryItem.js:389","message":"[LibraryItem] Library item \"0e52ea3a-5943-4e91-847f-a0d87c2722b6\" updated","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:05.427","source":"PodcastManager.js:239","message":"[PodcastManager] runEpisodeCheck: \"Pod Save America\" | Last check: Mon Apr 01 2024 14:00:07 GMT-0500 (Central Daylight Time) | Latest episode pubDate: Fri Mar 29 2024 02:00:00 GMT-0500 (Central Daylight Time)","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:05.750","source":"LibraryItem.js:389","message":"[LibraryItem] Library item \"5a9cdb7f-7191-4763-9675-f8ae51a34364\" updated","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:05.750","source":"PodcastManager.js:239","message":"[PodcastManager] runEpisodeCheck: \"What A Day\" | Last check: Mon Apr 01 2024 14:00:08 GMT-0500 (Central Daylight Time) | Latest episode pubDate: Mon Apr 01 2024 02:05:00 GMT-0500 (Central Daylight Time)","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 15:00:06.379","source":"LibraryItem.js:389","message":"[LibraryItem] Library item \"bde5f1d6-2ede-4404-9feb-daeb4ae98160\" updated","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:52.087","source":"Server.js:170","message":"=== Starting Server ===","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:52.088","source":"Server.js:101","message":"[Server] Init v2.8.1","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:52.315","source":"Database.js:178","message":"[Database] Initializing db at \"/config/absdatabase.sqlite\"","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:52.521","source":"Database.js:206","message":"[Database] Db connection was successful","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.104","source":"Database.js:167","message":"[Database] Db initialized with models: user, library, libraryFolder, book, podcast, podcastEpisode, libraryItem, mediaProgress, series, bookSeries, author, bookAuthor, collection, collectionBook, playlist, playlistMediaItem, device, playbackSession, feed, feedEpisode, setting, customMetadataProvider","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.262","source":"LogManager.js:66","message":"[LogManager] Init current daily log filename: 2024-04-01.txt","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.327","source":"BackupManager.js:260","message":"[BackupManager] 0 Backups Found","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.331","source":"BackupManager.js:57","message":"[BackupManager] Auto Backups are disabled","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.405","source":"Watcher.js:53","message":"[Watcher] Initializing watcher for \"eBooks\".","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.409","source":"Watcher.js:53","message":"[Watcher] Initializing watcher for \"Audiobooks\".","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.409","source":"Watcher.js:53","message":"[Watcher] Initializing watcher for \"Podcasts\".","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:53.446","source":"Server.js:319","message":"Listening on port :80","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:56.632","source":"Watcher.js:79","message":"[Watcher] \"Podcasts\" Ready","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:35:58.383","source":"Watcher.js:79","message":"[Watcher] \"Audiobooks\" Ready","levelName":"INFO","level":2}
{"timestamp":"2024-04-01 21:36:02.060","source":"Watcher.js:79","message":"[Watcher] \"eBooks\" Ready","levelName":"INFO","level":2}
nichwall commented 5 months ago

Having this issue as well. Based on info logs, mine seems to crash after updating podcasts. I've set the logger to debug and will post results when it crashes next.

Periodic crashes are unrelated to the MemoryStore warning. This warning is given as part of the startup logs and is not related to the crash. Please open a new issue for debugging the crash.

github-actions[bot] commented 1 month ago

Fixed in v2.12.0.

nichwall commented 1 month ago

Reverted in 2.12.2 to prevent the server becoming unresponsive after 24 hours. https://github.com/advplyr/audiobookshelf/issues/3251

skyzuma commented 2 weeks ago

if have a similar msg (Warning: connect.session() MemoryStore is not designed for a production environment, as it will leak) on unraid / docker / v2.12.3 ... i have no problems but i dont use podcasts ... so, idk if this msg important ...

nichwall commented 2 weeks ago

if have a similar msg (Warning: connect.session() MemoryStore is not designed for a production environment, as it will leak) on unraid / docker / v2.12.3 ... i have no problems but i dont use podcasts ... so, idk if this msg important ...

See the original message for this thread. This has been updated for the next release.

github-actions[bot] commented 1 week ago

Fixed in v2.13.0.