[Feature] Offline support

dskrad commented 2 years ago

@austinried What is the timeline for this feature? I love the app, but I am thinking of switching to something else so I can have offline (download music) feature.

austinried commented 2 years ago

It's coming... last update I included lots of backend changes that will support this (changes to the way images are fetched/cached to be more robust so the same system can support bigger files like songs). This next release will be bugfixes for that and i18n, and then I'm shooting for the release after that (most likely 1.4.0) for offline mode.

JohnXLivingston commented 2 years ago

It's coming... last update I included lots of backend changes that will support this (changes to the way images are fetched/cached to be more robust so the same system can support bigger files like songs). This next release will be bugfixes for that and i18n, and then I'm shooting for the release after that (most likely 1.4.0) for offline mode.

I'm testing your app. Works really great! Congrats! Can't wait for the offline mode!

174n commented 2 years ago

It should've been the first thing to get implemented for such an app. Such a good design, but unfortunately unusable for me

IngwiePhoenix commented 2 years ago

What's the current status on this?

hedgepigdaniel commented 1 year ago

Offline support is more or less working on 2.0.0! You can download albums and playlists, and there is an offline mode where the library view is restricted to downloaded music.

There are still some things I think are a not optimal:

When the app is opened with the internet down (Android airplane mode) and is not in offline mode, it does not get past the splash screen. The offline mode switch doesn't seem to persist between app runs. In combination, these issues make it impossible to start the app with the internet down.
It's not possible to browse the entire library with the internet down after being in offline mode. This is because the offline switch cannot be switched off until the server ping succeeds.
When browsing the library, the getArtistInfo2 and getAlbumInfo2 calls, and the requests to get artist/album images/thumbnails are made while browsing, and not upfront when syncing the library. Therefore when the internet is down and offline mode is off, thumbnails don't load consistently (also, they load slowly even when the internet is working).
It's not possible to mark things for downloading while offline, and I suspect that if there is any failure during the download process, the downloads will not be resumed.
Although you can mark an album/playlist for download, it is implemented literally as "download", when probably the most appropriate behaviour is to synchronize. E.g. when songs are added to an album/playlist on the server and that change is propagated to Subtracks, I think the newly added songs should be automatically downloaded if the playlist is available offline.

vbramselaar commented 1 year ago

Hi @hedgepigdaniel, I see you talking about the playlists. I don't see them at all and just thought that they were not implented yet. Am I wrong? I have playlists in Navidrome.

vbramselaar commented 1 year ago

Ow god now I see that you can swipe the music tab! @austinried planning to make this more clear maybe?

hedgepigdaniel commented 1 year ago

Some suggestions to address the above issues and improve the offline mode (@austinried I'd love your feedback on these, and I'd be happy to work on any of them):

Allow opening the app when the internet is down

Remove ping check for switching from offline to online mode
Fix whatever problem is preventing the app from opening when the internet is down

My mental model here is that offline mode is not the same as being offline. You can be in "online" mode whenever you want to browse the entire library, and you would choose offline mode when you want to play music and you don't have an internet connection. Both modes are usable when the internet is not available, except that music that isn't downloaded can't be played without internet. I expect that all the thumbnails don't consume significant space/data - but it could also be configurable whether to download them.

Include all library information in the database/cache, except for the audio

Include the information from getArtistInfo2 and getAlbumInfo2 in the database.
In the initial sync when adding a source, for all artists/albums, synchronize the above, all images/thumbnails, and any other data that is needed for anything except playing audio.
Separate the initial sync (when adding a source, or perhaps on an explicit "full sync" action), from the refresh action in the scrollable artists/album views. Doing O(n) requests for n artists takes at least a few minutes, so it's probably too slow for those use cases. Just calling getArtists/search3 et al (as the current sync service does) is enough to surface new/deleted artists/albums/songs, which is probably what the user wants in that case.

Advantages are that the thumbnails (and artist bio etc, if that is ever visible) will be available instantly (so no delayed image loading when scrolling albums/artists), and will be available offline to browse the entire library. Overall there would probably be less network requests, and none when the user is actively browsing - so maybe improved performance.

Save user preferences about what to download to the database, and download songs in the background

user can select albums, playlists, songs, and perhaps rules like recently played songs/albums, recently added music, etc - and mark them to be made available offline
download service downloads songs when a difference arises between the songs currently downloaded and those that should be downloaded according to the user's preferences.
download service downloads songs in priority order. e.g. first the play queue in order, then other things.
download service retries downloads when they fail, and resumes downloads when an internet connection becomes available
bonus: sync service periodically runs in the background, and therefore triggers the download of new music - e.g. if the user has configured that recently added music should be made available offline.

Advantages are that a user with an intermittent internet connection (e.g. in a car driving through rural areas), can browse the library and configure their play queue and what they want to be available offline, and Subtracks will handle downloading the necessary songs in the optimal order whenever it is possible.

Queue state changing actions to be performed when internet is available

Actions like starring, scrobbling, editing playlists should be possible when offline:

the effect should be visible in the app immediately
the action on the server should be delayed until the internet is available and perhaps retried a few times if it fails. I'd expect this would be implemented with a database table of such actions. For example to find whether a song was starred, the UI would check if the song is starred, and also if there is a queued action to synchronize the starring of the song.

austinried commented 1 year ago

@hedgepigdaniel Thanks a ton for all this feedback, it's really useful! Most of this is in line with what I am thinking for the direction offline support should go, so I think we are on the same page for the most part here. Some things I would like to clarify/go into detail on:

Both modes are usable when the internet is not available, except that music that isn't downloaded can't be played without internet.

This is good, but I would also like to have some sort of intelligent way to prompt the user to switch to offline mode when they have no internet access (or access to the server). I have started a bit of work on that by having a specific exception to throw when a source is offline, but haven't implemented anything yet. I still need an error messaging framework in general as well.

Include the information from getArtistInfo2 and getAlbumInfo2 in the database.

Right now I'm not using getAlbumInfo2, just getArtistInfo2 for artist images since the coverArt on artists hasn't historically been reliable, but I would like to revisit that strategy first. For the sync though, I am a bit worried about fetching all the images up front, even if they are just thumbnails, it seems like it could be quite a lot of data.

Currently there are two images for artists/albums/playlists: thumbnail and full size. Thumbnail is what you see browsing the library, and is either 256x256 or the "small" image URL from getArtistInfo2. Full is whatever size the server offers, which can be pretty big if you've got a lot of high-res artwork for albums. I'm not exactly opposed to trying to see what performance might be like grabbing all thumbnails up front, but I think a quicker strategy would be to fetch and save the info to at least have the image URL during sync (so calling the getArtistInfo2 if needed).

Save user preferences about what to download to the database, and download songs in the background

This part isn't what I had in mind for this, it's a bit more advanced, but I like it. It sounds easier for user to use/understand when changes (probably most often in playlists) need to propagate. I think this is the way to go for handling items marked as "downloaded," but it needs to be combined with some options about when to download songs in the background (on wifi, when charging, that kind of thing, ideally with conservative defaults).

I also appreciate the offer to work on some of these! At the moment if you'd like to start on your first one there, the two points under "Allow opening the app when the internet is down," that would be a great place to start I think. I'm not sure off hand what is actually stopping the app from starting when in airplane mode, but my guess is it's probably trying to fetch art somewhere during the initial init() when it shouldn't.

austinried commented 1 year ago

@vbramselaar Yes I would like to add some UI back to that view to indicate there are tabs there, I'm just not sure yet exactly how I'll do it. The main problem currently is that my ideal UI for that is tabs across the top that hide when you scroll, but I know that currently has a really annoying bug in flutter that is bad enough that I don't want to release with it: https://github.com/flutter/flutter/issues/40740

So, I'm still thinking about other ways to indicate tabs here.

ryansalerno commented 1 year ago

I apologize for chiming in with complications, but I would much prefer [Intentional] Offline Mode to not show me the full library. For very large libraries with small percentages of albums saved to my phone, scrolling forever for something I can actually play is overwhelming at best and not really an option when driving.

I suppose if everything were already synced to the app's DB as discussed above, this could be a fairly trivial setting to hide (or not) unavailable tracks (rolling up to albums and artists) in Library views. But in reading those suggestions, it struck me as somewhat unexpected behavior (that feels nice to have in some respects after sitting with the idea), but I would also be grateful for a naive Offline-is-Offline mode that didn't include a super comprehensive sync of info to the local DB.

hedgepigdaniel commented 1 year ago

I would much prefer [Intentional] Offline Mode to not show me the full library

To be clear, I don't want to change that - definitely offline mode should show only music that is available offline! All I'm suggesting above is for online mode to be less dependent on a continuous/perfect internet connection.

This is good, but I would also like to have some sort of intelligent way to prompt the user to switch to offline mode when they have no internet access (or access to the server)

@ryansalerno depending on your use case, I think this is probably a more relevant feature - that would make Subtracks more likely to show you the music that is playable currently.

I think there are use cases for automatic online/offline switch, and for manual modes:

driving, distracted, intermittent internet: Want offline mode to be on. Do not want online mode to engage automatically because it will make it harder to find stuff that is definitely playable.
on a train, intermittent internet, downloading music for a long trip. Want online mode to be on to access the entire library and choose things to download when the train goes past a town/reception. Don't want offline mode to engage automatically because it would interrupt the task of choosing music to download.
- listening to music, switching to airplane mode to save battery: want offline mode to engage automatically to not distract with music that isn't available. Vice versa when turning airplane mode off.
- walking down the street, playing one track at a time. Want online/offline mode to switch automatically if reception comes and goes so as to pick from all the tracks that can be played right now.

austinried commented 1 year ago

I'm thinking right now that only an automatic switch to or automatic prompt to ask to switch to offline mode would be best. There aren't many times you want to go from offline to online automatically, and I think in that last case switch back and forth between the two could be more confusing than helpful.

Given the potential to cause unexpected behavior though, it may be best to leave this feature for a potential 2.* feature based on feedback.

@ryansalerno no problem, I always appreciate the feedback. This is how it's working now when you're in offline mode (only showing downloaded) and I'd like to keep it that way as well.

trohnjavolta commented 1 year ago

Hi, is there a way to download everything with just one tap? That would be nice.

GabrielPearce commented 7 months ago

Hi, is there a way to download everything with just one tap? That would be nice.

Yes this would be awersome, could we also specify where locally we would like the files to download?

Ideally I would like to be able to use this as a way to have syncronise my local music with my server,

I use Audiobookshelf and it does this perfectly, would it be possible to have similiar behaviour in subtracks?

austinried / subtracks