search

jellyfin / jellyfin

The Free Software Media System
https://jellyfin.org
GNU General Public License v2.0
34.19k stars 3.12k forks source link

Collections affect library browsing #5424

Closed Liquoriced closed 3 years ago

Liquoriced commented 3 years ago

Enabling the 'Group movies into collections' option in the 'Library' settings in the Dashboard substantially slows down library browsing.

System

To Reproduce

  1. Go to Admin Dashboard
  2. Click on 'Libraries'
  3. Click on 'Display'
  4. Enable 'Group movies into collections' option
  5. Click 'Save'
  6. Click 'Home'
  7. Access movie library

Expected behavior Simply displaying the previously-saved movie collections should not introduce a dramatic slowdown when browsing movie library. Time to display first page of 100 movies ('Group movies into collections' option ENABLED): 9 seconds Time to display first page of 100 movies ('Group movies into collections' option DISABLED): Instantaneous

Time to display 'movie collections' tab: 14 seconds Time to display 'movie collections' tab when switching back a second time: Instantaneous

It looks like there's something wrong with how the movie collections are handled.

cvium commented 3 years ago

How many collections do you have?

Liquoriced commented 3 years ago

About 60.

cvium commented 3 years ago

I just installed the TMDb Box Sets plugin and it made ~45 collections for me. Loading the movie library is instant for me.

Does the log show anything? And what kind of hardware is Jellyfin installed on?

Silvereye89 commented 3 years ago

I just want to add that I have the same issue for about half a year now as well. Therefore it's likely not related to 10.7.0.

If I remember it correctly it started after the TmDb box sets plug-in has been fixed. Prior to the fix the plugin did not detect collections and even broke the existing Metadata.

So I am not sure if it is related to the plugin, an update somewhere in the 10.6.x or another coincidence.

I know that my answer might not be of great help for you now. But maybe some genuine people can narrow down the bug with this information.

Liquoriced commented 3 years ago

Yes, Silvereye89, the same issue was mentioned by someone else on Reddit a few months ago (https://www.reddit.com/r/jellyfin/comments/hy32oh/really_slow_to_see_catalog/). That's where I got the suggestion to disable 'group movies into collections'. I couldn't find it on GitHub under issues, hence I posted it here.

Opening the collections tab is slow, so that shows there's something wrong there.

cvium - the Jellyfin is installed on a Synology NAS (DS920+). It runs like a charm, with the exception of the collections issue.

I just saw that the log is referring to media files that have been removed: I'll delete all the collections and recreate them to see if the previous TmDb box sets plugin did indeed break the existing metadata.

Liquoriced commented 3 years ago

UPDATE: After deleting the collections, and deleting and reinstalling the TmDb box sets, I let the plugin build new collections. Unfortunately, the slow browsing has returned. Below is an example of the error given by the log.

[2021-03-12 03:48:09.150 +00:00] [WRN] [28] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from "http://192.168.1.101:8096/Users/f0d28b45a3284512a2aecaadae100aba/Items?SortBy=SortName%2CProductionYear&SortOrder=Ascending&IncludeItemTypes=Movie&Recursive=true&Fields=PrimaryImageAspectRatio%2CMediaSourceCount%2CBasicSyncInfo&ImageTypeLimit=1&EnableImageTypes=Primary%2CBackdrop%2CBanner%2CThumb&StartIndex=0&ParentId=ee593c21291143562d7d53c6566cc915&Limit=100" to "172.17.0.1" in 0:00:07.1437173 with Status Code 200 [2021-03-12 03:48:25.831 +00:00] [WRN] [23] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from "http://192.168.1.101:8096/Users/f0d28b45a3284512a2aecaadae100aba/Items?SortBy=SortName&SortOrder=Ascending&IncludeItemTypes=BoxSet&Recursive=true&Fields=PrimaryImageAspectRatio%2CSortName&ImageTypeLimit=1&EnableImageTypes=Primary%2CBackdrop%2CBanner%2CThumb&StartIndex=0&Limit=100&ParentId=ee593c21291143562d7d53c6566cc915" to "172.17.0.1" in 0:00:13.9185317 with Status Code 200 [2021-03-12 03:48:27.514 +00:00] [WRN] [31] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from "http://192.168.1.101:8096/DisplayPreferences/usersettings?userId=f0d28b45a3284512a2aecaadae100aba&client=emby" to "172.17.0.1" in 0:00:00.6098512 with Status Code 204 [2021-03-12 03:48:52.347 +00:00] [WRN] [23] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from "http://192.168.1.101:8096/Users/f0d28b45a3284512a2aecaadae100aba/Items?SortBy=SortName%2CProductionYear&SortOrder=Ascending&IncludeItemTypes=Movie&Recursive=true&Fields=PrimaryImageAspectRatio%2CMediaSourceCount%2CBasicSyncInfo&ImageTypeLimit=1&EnableImageTypes=Primary%2CBackdrop%2CBanner%2CThumb&StartIndex=0&ParentId=ee593c21291143562d7d53c6566cc915&Limit=100" to "172.17.0.1" in 0:00:07.1344774 with Status Code 200

Nickbert7 commented 3 years ago

I can confirm this issue. (7090 Movies with 519 Colletions, database on a SSD) It is amazing how fast it is getting by disabling the option when you had enabled the feature for a long time. 😂

Silvereye89 commented 3 years ago

I've just updated to 10.7.1 (Docker of linuxserver/jellyfin) and noticed that the time to open the movie library has even increased slightly.

I am also having the same log warnings like Liquoriced. Additionally I've noticed two movies, that I've deleted about a year ago, report issues in the log file at the same time. I don't know if this might be related. I tried deleting these entries manually in the respective collections.xml. But the the log warning still occurs.

[2021-03-24 10:05:41.185 +01:00] [WRN] [41] MediaBrowser.Controller.Entities.BaseItem: Unable to find linked item at path "/data/Movies/_Example_Collection/Movie_1/Movie1.mp4" [2021-03-24 10:05:41.186 +01:00] [WRN] [41] MediaBrowser.Controller.Entities.BaseItem: Unable to find linked item at path "/data/data/Movies/_Example_Collection/Movie_2/Movie2.mp4" [2021-03-24 10:05:57.131 +01:00] [WRN] [41] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from "https://_MyDomain.tld_/Users/_UserID_/Items?SortBy=SortName%2CProductionYear&SortOrder=Ascending&IncludeItemTypes=Movie&Recursive=true&Fields=PrimaryImageAspectRatio%2CMediaSourceCount%2CBasicSyncInfo&ImageTypeLimit=1&EnableImageTypes=Primary%2CBackdrop%2CBanner%2CThumb&StartIndex=0&ParentId=7a2175bccb1f1a94152cbd2b2bae8f6d&Limit=100" to "192.168.1.1" in 0:00:16.8225297 with Status Code 200

cvium commented 3 years ago

To get a better understanding of this I think we need the following:

  1. Server hardware
  2. Server version
  3. Number of movies
  4. Number of collections
  5. Are any movies in multiple collections?
  6. How many movies have multiple versions?
Silvereye89 commented 3 years ago

Thanks for your reply, Cvium!

1. Hardware: Synology NAS 918+ with 8GB RAM, running Jellyfin in Docker via linuxserver/jellyfin container 2. Server version: 10.7.1 3. Number of movies: 800 4. Number of collections: 90 5. Are any movies in multiple collections? Yes, I created an extra collection for some Western movies. This collection also contains movie series (collections). 6. Multiple versions of movies? None

Thanks in advance for investigating here.

kick224 commented 3 years ago

I can confirm the long loading times. I tested multiple sizes and on my system the library stops loading entirely after around 3000 movies. I kept the page open for 3 hours but there is only the loading symbol. Interestingly, even with the biggest size (~4000 movies), the library still loads in around 3 seconds if you select a filter (like Features->Subtitles, ~3900 movies found)

1. Hardware: Raspberry Pi 4 with SSD. All data except for the movies itself is on a SSD. 2. Server version: 10.7.1 3. Number of movies: ~4000 4. Number of collections: 610 5. Are any movies in multiple collections? None 6. Multiple versions of movies? None

fuzxi commented 3 years ago

My experience: near-instant load time of my Movies library (<1 second) with "Group movies into collections" disabled. 9 second load times with it enabled, even with library page size set to 1. My load time for Collections is somewhat long regardless of this setting (around 5.5 seconds). None of my collections have more than 8 items.

  1. Hardware: Raspberry Pi 4 2GB with SD card.
  2. Server version: 10.7.2
  3. Number of movies: 139
  4. Number of collections: 19
  5. Are any movies in multiple collections? No
  6. Multiple versions of movies? Two copies of one movie, displayed as separate movies. Both are in one collection each (not the same collection).