search

tehkillerbee / mopidy-tidal

Tidal Backend plugin for Mopidy
Apache License 2.0
93 stars 28 forks source link

Performance issues with 200+ playlists #90

Closed DavisNT closed 2 years ago

DavisNT commented 2 years ago

Mopidy-Tidal 0.3.1 has performance issues on Raspberry Pi 3 with Tidal account having 200+ playlists (each playlist on average has around 30 songs). It takes around 40 seconds for any web extension to list (refresh) the playlists. Entries stating mopidy_tidal.playlists Calculating playlist updates.. are logged at the beginning and end of those 40 seconds.

Could it be possible to refresh the playlists asynchronously (return cached playlists and do the actual refreshing in background)? Maybe this could be a configurable option...

blacklight commented 2 years ago

This should be addressed (or at least mitigated) in https://github.com/tehkillerbee/mopidy-tidal/pull/92.

Unfortunately, the refresh can't be asynchronous. The backend can be more lazy when it comes to loading actual playlist items, but when you actually load the list of playlists, the client needs to render the list within the stack of the lookup call. And that's where things are slow in your case - a high number of playlists means more data sent down the wire, and more API calls to manage pagination.

The existing logic mitigates this overhead with a "grace period" of 5 minutes - meaning that if you reload the list within 5 minutes, the cached results should be retrieved and the backend won't make new calls (this is useful in case playlists are modified in other clients, so you won't have to reload mopidy to get the changes). The new PR makes this logic configurable. By setting playlist_cache_refresh_secs to zero you will load the list of playlists only at startup, and it won't be updated until you restart mopidy. This is a bit similar to the logic implemented on mopidy-spotify, with the difference that the initial cost is spent on the first lookup, not on startup. Any other number of seconds will establish how long the list of playlists should be cached before it's synchronized with the upstream API.

tehkillerbee commented 2 years ago

Fixed in #92

DavisNT commented 1 year ago

Thank you! And sorry for the late response!

Do I understand correctly that setting playlist_cache_refresh_secs to zero would make the playlists to be loaded at Mopidy startup (not during first lookup)?

blacklight commented 1 year ago

Not at startup - that's what mopidy-spotify does, and IMHO it eats away a lot of initialization time. It still does a lazy initialization when playlists are looked up the first time, but after the initialization it won't refresh the playlists until mopidy is restarted.

DavisNT commented 1 year ago

Thanks! I see.

I am not sure whether functionality like search in Iris doesn't read all the playlists at once (that would lead to notable delay if playlists get refreshed/loaded)?