search

OxygenCobalt / Auxio

A simple, rational music player for android
GNU General Public License v3.0
2.15k stars 143 forks source link

Service independence (Starting playback without opening app) #244

Closed etyarews closed 6 months ago

etyarews commented 2 years ago

Describe the feature you want to implement:

While Auxio have Intents, they are limited and don't work unless Auxio is already running in the background. My knowledge of programming is limited, but it appears the latter is caused by the type of intent.

Is your feature request related to a problem? Please describe:

I'm trying to control music playback through Auxio, namely I'm a user of Tasker and sometime between Android 7 and Android 12 it became unreliable to start music playback by just simulating a Media Button press. Intents appear to be the only solution left, and sadly the ones Auxio have are of the broadcast type.

Do other music players handle this? If so, how?

PowerApp actually does have it, they are of the Service type and I'm able to start them even after a reboot, and it has a long list of possible intents that I'm aware of:

Why do you think this will improve everyone's usage of Auxio?

While not useful to the majority of Auxio users, it will be incredibly helpful to the ones that are also Tasker users, and because it isn't a request for a Tasker Plug-in, it has could help users of that can send Intents.

Due Diligence:

OxygenCobalt commented 2 years ago

This is not possible.

To make my app work in modern android, I am effectively forbidden from having any means of interaction that does not require you to open the app first.

Android 8 made foreground services (Which includes Auxio's music playback) MUCH harder to initialize. I have 5 seconds to initialize everything or the app will crash. Factoring in music loading and the restoration of the playback state, I simply cannot fit it into that window.

The options I have to work around it are:

Even then, I also have to deal with Android 12's insane restriction on launching foreground services when the app is backgrounded. If I were to add these intents, it could open up a footgun where someone on Android 12 tries to fire an intent while Auxio is backgrounded, and then gets confused why the app crashes. It just isn't practical.

The days of being able to do neat things without interaction on Android are over. I do not want my app to be an awful, buggy mess, so I require you to open Auxio before doing anything. If this means I have to annoy a vanishingly small minority of Tasker users, so be it.

Besides, the MediaController API is really what one should be using to control playback, not Intent instances. Auxio does support a variety of actions with that API, save queue editing and browsing the music library (I'm working on that). The Tasker developers should really add support for that instead of trying to salvage a dying API.

etyarews commented 2 years ago

Hey if you don't mind just a side note, so I don't have to open another issue for something that can be easily answered:

What about Static Shortcuts? Auxio has a Dynamic one that doesn't show up both in Tasker and Kustom, and other non-launcher apps that can initiate shortcuts. I've managed to in a really dumb roundabout way trigger the Dynamic one with Tasker, but it would be far easier and more flexible if it was possible for you to implement a couple Static ones(or even just one). Again, this would also benefit Kustom users, and others.

I also think Tasker is already using the MediaController API, as we can get Media information and interact as if it was media buttons as I stated. But even if I'm wrong, the google documentation mentions its only for ongoing media session, I don't think it is possible to start a media session from nowhere without initializing the app first.

OxygenCobalt commented 2 years ago

What about Static Shortcuts? Auxio has a Dynamic one that doesn't show up both in Tasker and Kustom, and other non-launcher apps that can initiate shortcuts. I've managed to in a really dumb roundabout way trigger the Dynamic one with Tasker, but it would be far easier and more flexible if it was possible for you to implement a couple Static ones(or even just one). Again, this would also benefit Kustom users, and others.

I use dynamic shortcuts so that shotcuts in debug builds don't redirect to release builds or just not work at all. Maybe I can do some cursed hack where the release shortcut is static and the debug shortcut is dynamic.

I also think Tasker is already using the MediaController API, as we can get Media information and interact as if it was media buttons as I stated. But even if I'm wrong, the google documentation mentions its only for ongoing media session, I don't think it is possible to start a media session from nowhere without initializing the app first.

Yeah, I already know. I brought it up since controlling it through MediaController would not unexpectedly initialize the app, which was impossible as I stated prior.

OxygenCobalt commented 2 years ago

Okay, let me clarify that this might not be entirely impossible. It's just impossible with the current system and not something that I actively want to say I have planned.

Over time I tend to warm up to features as I think on and off about them and eventually develop a system that will solve the problem sensibly. If I'm confident that the odds have shifted and the addition is a net benefit with the new approach, I create a new issue with my proposal and signal that it's actively planned and not just something I'm thinking about.

For example, I dismissed using my own metadata extractor/caching mechanism (i.e Ignore MediaStore tags) until I found a sensible way to implement it (Using ExoPlayer instead of JAudioTagger and caching raw tags instead of the entire music library), and then I was comfortable implementing it in the app.

For this one, I may revisit it if I find a coherent way to initialize everything that does not negatively affect the app experience as a whole.

etyarews commented 2 years ago

Thank for your consideration, but just to clarify: are you talking about intents or static shortcuts?

OxygenCobalt commented 2 years ago

Intents. The static shortcuts thing I need to do more research on but should be slightly more possible.

OxygenCobalt commented 2 years ago

Okay, turns out your feature request might be required for #37 because the MediaBrowserService lifecycle is absurd. I haven't looked into it very much though.

etyarews commented 2 years ago

So it's going to get open again?

I will not lie, I don't totally understand the technical implications or even why it is required for Android Auto, but hey, it is still an improvement

OxygenCobalt commented 2 years ago

Maybe. The documentation is extremely vague, but it implies that Android Auto can start the playback service at any time, including when the app is completely dead. This is the same avenue that you want with your intents, except I might be forced to implement it.

etyarews commented 2 years ago

I never even looked at Android Auto, but thinking about it, it doesn't make sense to have to manually open a Music Player while in a car.

Best case would be to test and see if a Google app can auto start with Android Auto.... except last I checked, Google doesn't make any music player anymore

OxygenCobalt commented 2 years ago

Youtube Music exists. I'm going to add the media browser boilerplate and see what happens in an emulator.

Assuming that I need to handle this case, I think I have a route to implementing it. Basically, I would need to unify the two services in Auxio (music loading and playback) into a single service that can load and play music which resolves the issues with foreground time limits and app initialization in general. Albeit, I'd want to do it in such a way that the code for music loading and music playback remain separated, as it's easier to maintain in that way.

The implementation details are hard and annoying and blocked by countless things however, so this is still low-priority.

etyarews commented 2 years ago

If you don't mind me asking, why is Auxio so aggressive with music loading? I don't think I've ever seen an app being so aggressive and upfront about checking for changes to the library.

  1. It reloads it every time you start the app, even if it was seconds since the last time it checked it
  2. It doesn't happen in the background, meaning it stops the user from using the app.
  3. "Ignore MediaStore Tags" takes significant time to reload (which is fine), but because of the above points it isn't really a viable option.
OxygenCobalt commented 2 years ago

A few reasons, I guess:

  1. I used to use Music Player GO, and so a bunch of Auxio's architecture is derived from that app. MPGO is similarly aggressive with loading.
  2. Being aggressive with loading ensures a more sensible music experience at the cost of lost time at the start. I've seen how background loading mechanisms work in other apps (Read: Vanilla) and they are buggy garbage that usually renders the library similarly unusable until it's done loading. I'd rather be upfront that "Yeah, I'm is unusable until I can index your 10000-song library, sorry" rather than be implicitly unstable until the loading is done.

Note that the next version will have a much faster music loader where "Ignore MediaStore tags" is on by default, as I am now caching song metadata for later use. There will likely always be some overhead though.

etyarews commented 2 years ago

Eeeh, I'm going to have to look at the next version first to give it some thought and see what really changed in terms of the user experience, because I'm not sure about the implications of caching song metadata without seeing it in action.

But I don't really get why it isn't doing like VLC by doing a big scan on boot and on the background, an option to manually refresh the library on the 3-dot menu would also help. The probability of the user changing the music library (specially if they are using a whitelist of folders) and not remembering it by doing a manual re-scan is rather low. Furthermore, you could mitigate the number of auto-refreshes by comparing when was the last time it was done, as it redo it even if you closed and reopened the app in less than 5 secs, but I'm assuming this will be taken care of with the caching.

OxygenCobalt commented 2 years ago

The caching implementation is mostly to circumvent the time sink within "Ignore MediaStore tags", that being the quite slow file opening/metadata extracting process. It works something like this:

  1. Query the media database for every song on the device.
  2. For each song, check with the cache if it has extracted metadata for that song that can be used. This is used by comparing the timestamps of the song from the media database and the timestamps in the cache.
  3. If we can use the cached metadata (the timestamps are the same, so the song has not changed), we are done.
  4. Otherwise, we run the slower metadata extraction routine and save that song to the cache.

So, if you don't change your library, loads should go from taking ~15 seconds for a 1k song library to only 1 second or lower. If you only change part of your library, then it should still be quite fast, with a bit more time being needed to scan the new music.

But I don't really get why it isn't doing like VLC by doing a big scan on boot and on the background

AFAIK, newer android versions will not allow me to wake my app on boot anymore unless the user disables battery optimizations. I really want the app to work out of the box without having to make the user fiddle with hidden android settings, so I can't implement this.

The probability of the user changing the music library (specially if they are using a whitelist of folders) and not remembering it by doing a manual re-scan is rather low.

Not really sure what you mean by this.

Furthermore, you could mitigate the number of auto-refreshes by comparing when was the last time it was done, as it redo it even if you closed and reopened the app in less than 5 secs, but I'm assuming this will be taken care of with the caching.

That's basically the cache mechanism, albeit done on a song-by-song basis so it still takes a non-trivial amount of time. Even if I did some clever tracking of when loads occur, you also have to consider that I still have to load the cache as well, hence why I said there will always be some overhead.

etyarews commented 2 years ago

The caching implementation is mostly to circumvent the time sink within "Ignore MediaStore tags", that being the quite slow file opening/metadata extracting process. It works something like this:

1. Query the media database for every song on the device.

2. For each song, check with the cache if it has extracted metadata for that song that can be used. This is used by comparing the timestamps of the song from the media database and the timestamps in the cache.

3. If we can use the cached metadata (the timestamps are the same, so the song has not changed), we are done.

4. Otherwise, we run the slower metadata extraction routine and save that song to the cache.

So, if you don't change your library, loads should go from taking ~15 seconds for a 1k song library to only 1 second or lower. If you only change part of your library, then it should still be quite fast, with a bit more time being needed to scan the new music.

Yeah, the caching part seems fair. However, if it is really 1 second or lower, you might want to consider "hiding" it behind a fake app loading screen that takes half to one second to complete and the rest of the loading is shown as usual. This way, the app will appear faster than it actually is.

But I don't really get why it isn't doing like VLC by doing a big scan on boot and on the background

AFAIK, newer android versions will not allow me to wake my app on boot anymore unless the user disables battery optimizations. I really want the app to work out of the box without having to make the user fiddle with hidden android settings, so I can't implement this.

Alright, so I'm on Android 12, but there are 2 things which might minimize your concerns:

  1. Tasker can show a system floating dialog that directly asks you whether you want to disable battery optimizations.
  2. Kustom apps show a floating dialog that concisely tells the user how to disable battery optimization, and it takes the user directly to the android setting screen, so it's not hidden.
  3. Both apps somehow can know if the battery optimization was disabled or not, re-enabling it makes those apps ask again for the user to disable it.

The third point is important, because you can use it as a means to fall back to the previous method. Or make the on boot an option in the settings that only nags the user when they open the app for the first time.

The probability of the user changing the music library (specially if they are using a whitelist of folders) and not remembering it by doing a manual re-scan is rather low.

Not really sure what you mean by this.

AFAIK Auxio works in two "modes":

If the user is using the "Whitelist" method, they are more likely to be aware of changes they made to those folders, meaning they are capable of manually triggering a re-scan, if there was an option on the 3-dot menu AND scan only happened on boot.

Basically, "Whitelist-users" would be fine if scan only happened on boot.

OxygenCobalt commented 2 years ago

Yeah, the caching part seems fair. However, if it is really 1 second or lower, you might want to consider "hiding" it behind a fake app loading screen that takes half to one second to complete and the rest of the loading is shown as usual. This way, the app will appear faster than it actually is.

Let me clarify that it's 1 second or so for an "average" library (1000 songs), larger libraries will slow the loading process down more. Because of this, I may as well keep the loading UI as-is.

Alright, so I'm on Android 12, but there are 2 things which might minimize your concerns:

But that's the thing: I don't want to have to make an annoying onboarding flow just to disable battery optimizations. I want my app to work without much fiddling by the user. No matter how obvious I make the dialog, no matter how much I nag the user, some user will inevitably ignore it and then make an issue complaining that the app is crashing. "Make something dumb-user-proof and the universe will make a dumber user."

Besides, disabling the optimizations is the "easy way out" that will only likely get harder and harder in future versions.

etyarews commented 2 years ago

Let me clarify that it's 1 second or so for an "average" library (1000 songs), larger libraries will slow the loading process down more. Because of this, I may as well keep the loading UI as-is.

I know, that's not what I mean.

Assuming:

  1. Cached loading will take between 0.5 to 1 second to load a library with 1000 songs
  2. You know with certainty that the vast majority of users don't keep more than 1000 songs on their devices.

You can then make a fake full screen splash screen on top of the loading dialog. It is dismissed if:

In this case, most users wouldn't even know Auxio is loading the library. The ones that have bigger libraries will have Auxio open into a loading dialog that is already in the halfway point, meaning that for them, it will also appear like Auxio is faster than it actually is.

Granted this adds complexity, and I'm not even 100% into the idea, but eeh, I needed to explain a little since Loading Screens are really annoying for end users, specially ones that appears every time you cold start the app and don't allow you to interact with the app while it is loading.

Besides, disabling the optimizations is the "easy way out" that will only likely get harder and harder in future versions.

I understand your point, and can't really deny them, however, I'm not 100% sure about this point in specific. Android 13 added a convenient way to check for active apps in the notification drawer, this seems to point to me that Google is considering backtracking a bit on the battery limitations as long as they find an easy way to communicate it to the user

OxygenCobalt commented 2 years ago

I know, that's not what I mean ...

I see. Doing that is actually not recommended by google anymore. The splash screen is only intended for basic app initialization. Otherwise for loading I should use placeholders or indicators, like I currently am. If I were to shove in my own splash to hide loading, that would just create needless jank that I don't want.

Besides, disabling the optimizations is the "easy way out" that will only likely get harder and harder in future versions.

The FGS is merely another way to cripple apps under the guide of "battery protection". You now have the free ability to completely kill an app (not just the service, the WHOLE APP) if it has an active foreground service, making services even harder to manage. I don't even have the leeway to save the playback state or reset the widget when it's pressed, my app is just killed. Of course, the real reason behind most service limitations is to railroad apps into proprietary GCM/firebase garbage, so the FGS is a logical step in that to make typical monitoring or data sync services even less usable.

OxygenCobalt commented 2 years ago

Okay, I'm going to re-open this. I realized that if I unify my two services, that more or less resolves the issues of background starts and the foreground time limit. It's hard, but doable, and it will actually enable #37. I think I'll try to do it alongside #257 in a more general refactor of the service lifecycle.

OxygenCobalt commented 1 year ago

Actually, may not necessarily need to unify the music loading and playback services. Seems like the type of work the music loading service is doing will be deprecated in future versions. To futureproof it, I'll have to use the new "work manager" system. No clue how much easier or harder this becomes.

etyarews commented 1 year ago

A bit of off-topic, but since this conversation started because of it, could you consider adding a Tasker Plugin?

The Tasker dev made a video explaining how to do it: https://www.youtube.com/watch?v=48IVJgDtu6Y

And the documentation seems... relatively easy enough

OxygenCobalt commented 1 year ago

I can, but it's still blocked by this issue since Auxio physically cannot work correctly with Tasker without this change.

etyarews commented 1 year ago

Don't the shortcut implies that Auxio already has the means to work if an app manually calls it?

As I said a while ago, what prevents me from using Auxio in Tasker is because it uses dynamic shortcuts, even then, I managed to use a truly cursed method (a hell to set-up and configure) to make Trigger the Shuffle Dynamic Shortcut and it worked.

OxygenCobalt commented 1 year ago

Not quite. The shortcut opens the app's activity, which then delivers commands to the services. What I'm talking about is the ability to cut the app activity out, and simply deliver the command to the service directly. Auxio doesn't really support this right now.

Oh, also scrapping the "work manager" thing I brought up earlier. Turns out it's too OEM-dependent and once again runs into the same foreground start limitation insanity as prior. Just going to lie and tell the system that the unified service is solely for "media playback" when in reality it's doing playback and loading.

etyarews commented 1 year ago

Would it be possible to just open a dialog or something and immediately close it?

OxygenCobalt commented 1 year ago

Too janky of a solution for something that can be solved in other ways.

etyarews commented 1 year ago

I mean, everytime we open Auxio, it opens a dialog about updating the library anyway.

Even if you found a perfect solution that can start playback from cold boot, it would still need to load the music library, no? By that point you would need to create a notification or a dialog anyway

OxygenCobalt commented 1 year ago

Oh, that's what you mean. Yes, I would have to show a notification. The issue is that the service performing playback and the service loading music are separate right now. Arguably, I could make the playback service show a fake notification to work around the limitation as it waits for music to load, but that's honestly too bug-prone when I can also just merge the two services and avoid such.

OxygenCobalt commented 1 year ago

Regarding our discussion from awhile back, #384 is actually likely going to make it that Auxio will actually stop indicating that it's loading music front-and-center in some cases. It's still in flux however which cases Auxio will no longer indicate it, however. Once I pin it down I'll share the specifics.

etyarews commented 8 months ago

It's been 84 years. Any update on this?

Just to refresh, what I actually want is either Tasker Plugin implementation, or, a static shortcut that can play music through Auxio.

OxygenCobalt commented 8 months ago

Okay @etyarews, so I haven't been able to do much regarding this is because of:

  1. School
  2. Playlists
  3. More School

You can become a sponsor though and prioritize this issue, making me work on it first before anything else (No guarantees on me implementing it by some date though, again because school). An $8/month sponsor has already done that with #342, so I'm doing that first unless a $16/month sponsor prioritizes something else. After that, it'll either be #322 or this issue, depending on which one I want to do.

OxygenCobalt commented 8 months ago

This has been prioritized by @gtsiam, @etyarews. I will be implementing it after #342. If you also prioritize it, I'll do it before #342.

etyarews commented 8 months ago

I'm sponsoring it for now. Can't do more cause exchange rates aren't really helping me right now. Mark this as doubly prioritized

OxygenCobalt commented 8 months ago

Yep, this issue now has $12 backing it, so it will be done first @etyarews.

OxygenCobalt commented 8 months ago

I'm making slow, incremental progress on making this work. The playback and music services are now combined, but now I have to operate under the assumption that I can never briefly exit foreground whatsoever. This is really hard and it's going to take awhile for me to figure out the state.

Question: How are you going to send commands to Auxio through tasker @etyarews? I don't want to pay for Tasker to figure this out. Is it a broadcast, a service start call, or direct operation of the media session?

etyarews commented 8 months ago

Alright, so here's the documentation about making a Tasker Plugin:

https://tasker.joaoapps.com/pluginslibrary.html

There's roughly three concepts:

  1. Action: Tasker will do something to the app. I.e. Tasker will trigger Auxio to play, either the current playlist, or a song, or another playlist.
  2. Event Condition: Something happened in the app. I.e. Auxio started playing
  3. State Condition: Something is/isn't happening to the app. I.e. Auxio is playing(which implies Tasker can know when it is not)

I imagine static shortcuts are reasonably similar to what an Action could be in Auxio, so you might want want to start with that.

In an ideal world I'd want to be able to control Auxio entirely with in Tasker. Playing, Skip, Previous, Stop, Play specific file/playlist, command to play on specific time, etc..

OxygenCobalt commented 8 months ago

Thanks for that @etyarews. I'm still not fully sure how to route the intents to PlaybackService. I'll figure that out eventually.

Anyway here's a progress update:

So I've merged the two services, kind-of. What used to be the services now inherit a weird "fragment" class I made up. But they are still operating under the assumption that the app is in the foreground. The issue is that the flow might be something like:

And that's if there's music to be loaded and things to play. If there's neither, no foreground will occur and eventually the app is killed by the system for not starting foreground in time. As a result, I must design the service lifecycle under two hard rules:

  1. Absolutely no downtime. Even if you're not doing work, hang on the last foreground notification until you're absolutely certain we can safely exit foreground without suprises. I think I can do this by basically making the service "fragments" flag if they want to foreground, background, or if it doesn't know what it wants to do yet.
  2. Always assume you have basically no time to go foreground. OEMs probably play fast and loose with the foreground time limit, so I may have to start foreground with a fake notification just to satisfy the OS.

There's likely more refinement I need to do regarding this. I have some ideas on how to detect from the service's end if it was started by another part of the OS or if it was started by the app, which should allow me to switch between uptight and loose foreground behavior. Sadly I'm tied up with all the 3.4.0 bug reports, school, and a hackathon, so I'm not going to be able to do much for now.

etyarews commented 8 months ago

And that's if there's music to be loaded and things to play. If there's neither, no foreground will occur and eventually the app is killed by the system for not starting foreground in time

So, you can start the service in the background, the issue is to keep it in the background?

OxygenCobalt commented 8 months ago

Basically, you can start a service by:

  1. startService, which will lock you into the background. If you go foreground, android will stop you.
  2. startForegroundService, which let's you go into foreground but ONLY if you go foreground in ~5 seconds (but this varies). Otherwise you'll crash for not foregrounding in time. I assume that every foreign component that starts Auxio will be doing it this way.

@etyarews

etyarews commented 8 months ago

Wait, what happens, currently, when you start the app, start playback, go to home and then remove Auxio from the recents screen? Isn't that changing from foreground to background anyway?

OxygenCobalt commented 8 months ago

@etyarews

OxygenCobalt commented 8 months ago

The only current circumstance in the app where the service feasibly goes from foreground -> background -> foreground is when playback is stopped while the activity is in recents, and then a media button event starts playback again. I think that's an exception though since it's a broadcast. I might have to confirm that though or otherwise my entire idea of how to go about this change is flawed.

etyarews commented 8 months ago

Alright, so I've glanced around android docs and it appears that a foreground service needs to start a notification or something in order to count as "foreground" is that right?

OxygenCobalt commented 8 months ago

Yeah, going foreground requires a notification to indicate such.

etyarews commented 8 months ago

Yeah, but isn't this solved?

  1. start foreground service
  2. Create notification showing the library is loading
  3. Start playback
OxygenCobalt commented 8 months ago

Number 2/3 assumes that music hasn't already been loaded and playback hasn't occurred, and does not handle the possible gap that occurs in foreground state between 2 and 3. @etyarews

etyarews commented 8 months ago

Isn't the point to merge the two states into one that can handle both?

OxygenCobalt commented 8 months ago

Yes, but to do that necessarily means that those gaps will arise and must be handled @etyarews. The service unification solves the problem of the playback service needing to spin up the music service (not possible if app/service is in background).

etyarews commented 8 months ago

If it is the same service, wouldn't there be no actual gap for Android to see? Like, I understand there is a gap to execute it, but it is the same service just waiting for code to finish, no?