Dealing with background restrictions on non-stock android phones

shankari commented 4 years ago

Since e-mission started as a research project, all the initial development was on stock android, primarily running on Nexus phones. Since I wrote the code to be light on background usage, it doesn't use a wakelock. Instead, it uses an intent service that receives messages from the location APIs.

After API 28, the intent service is registered as a foreground service so it is allowed to receive fine grained updates.

As e-mission becomes more widely adopted, though, we need to deal with non-stock android phones. There are many known issues with non-stock phones and background operation (https://dontkillmyapp.com/) and all of them require manual user intervention to allow the app to work properly.

This issue tracks the question of how to let the users know what they need to know.

shankari commented 3 years ago

For the record, on the Samsung Galaxy Edge, we need: (1) app is not sleeping (Settings -> Device maintenace -> Battery -> Unmonitored apps) (2) app is not optimized (Settings -> Apps -> Special access -> Optimize Battery Usage) (3) Power savings is off (Settings -> Device Management -> Power)

Unfortunately, both 1 and 3 seem to be required. If I change only (3) to “Mid” instead of “Off”, the trips get wonky again. The trip on the left was with (3) set to “Mid” and the one on the right was with it set to “Off”. No other changes.

Power savings = Mid	Power savings = Off

PatGendre commented 3 years ago

@shankari thanks ! Useful to know, I Guess this is similar for other models : Set all battery/energy settings to minimum.

shankari commented 3 years ago

@PatGendre right, but in some of the other makes/models, it is sufficient to set this particular app's battery/energy settings to a minimum. For the Galaxy Edge, we have to set the entire phone's battery/energy to a minimum. It does seem to be required - if I just flip it to medium, the trip quality degrades significantly, even if the app is itself not optimized.

PatGendre commented 3 years ago

Hi @Shankari , we had a meeting this morning with the team from Lyon testing e-mission with cozycloud. They also felt that the background restrictions are a real concern for Android users. They have the impression that problem could be mitigated if relying less on Geofence for detecting trip starts, and maybe more on the ActivityDetection API and their intern will try to work on this.
What's your experience with geofence vs activityDetection wrt android restrictions, could an alternative to Geofence solve in part the issue?

shankari commented 3 years ago

@PatGendre I've wanted to explore activity detection (in particular, the new activity transition API on android) for detecting trip starts for a while, so it is exciting to see that a member of the community will work on it. If it works, I hope that they will contribute the code back to the project.

However, I regret to inform you that it will not actually solve all problems with background operation that we encounter, unless you are willing to drop the requirement to read locations. Please see my notes in https://github.com/e-mission/e-mission-docs/issues/535#issuecomment-848390001 and the subsequent comments for the default settings with continuous location tracking turned on. In continuous location mode (Developer Zone -> Collection -> Duty Cycling) we do not detect trip start or end, we just keep requesting the location continuously.

I experimented with it to see whether we could just leave location on all the time from the app, and let the OS manage the location turn on/off. But unless the app is prevented from "sleeping", the OS stops giving the app location values, even when we are in motion.

In particular, see https://github.com/e-mission/e-mission-docs/issues/535#issuecomment-849273585, which seems to indicate that the app is running, but the GPS provider is turned off.

05-26 11:19:00.500  3642  9487 D LocationManagerService: provider request: gps ProviderRequest[ON interval=+30s0ms]
05-26 11:30:35.366  3642  3654 D LocationManagerService: provider request: gps ProviderRequest[OFF]
05-26 11:50:37.094  3642  3653 D LocationManagerService: provider request: gps ProviderRequest[ON interval=+30s0ms]
05-26 12:39:22.715  3642  3911 D LocationManagerService: provider request: gps ProviderRequest[OFF]

This may be specific to the Samsung phones, but it at least shows that switching to activity recognition is not a magic bullet that will solve everything.

We can actually test quite a bit of functionality around location with the current app, without writing code, by changing the settings in the developer zone. I would suggest that the intern start with that on the phones that have problems so we know that changing the code will be worthwhile. I'd be happy to meet with the team in Lyon to discuss further.

PatGendre commented 3 years ago

@shankari thanks for your quick and information rich reply, I will transmit of course :-)
(As far as I'm concerned I will leave for vacations and give you news back around mid August ;-)

shankari commented 3 years ago

Happy vacation! I envy the European 3-week summer vacation! Feel free to give them my direct email so they can contact me when you are gone (if necessary).

shankari commented 3 years ago

For the record, while helping the CanBikeCo participants with installs, I noticed that closed source apps for mileage tracking have similar instructions in their FAQ.

https://support.onfleet.com/hc/en-us/articles/360035454452--Battery-Optimization-Background-Restrictions#h_dad5c3ec-6ffd-4135-90b6-3ef7f2847f98 https://university.hurdlr.com/en/articles/4460541-lg-background-restrictions-that-cause-mileage-issues

Based on purely anecdotal evidence (n=2) if you tell people that the problem is with background restrictions, and they are motivated to run the app, they are able to figure it out (go to the app and power screen, look at all settings and disable settings related to power).

Abby-Wheelis commented 1 year ago

We have some concerns about Samsung again! Copying over discussion from duplicate thread for continuity:

See prev discussion: phone#1009, pulling the discussion that started in this comment, since it's clear this requires some investigation and experimentation.

Starting point: In the process of the Profile Settings migration into React, I've made a rough draft of the Permissions as a React Modal. In writing this, we've come across some code related to Samsung-specific permissions that isn't currently displayed, so we're investigating how well the app works on a Samsung device. Samsung is the worst-rated permissions blocker on dontkillmyapp, so we know there could be issues other Android devices don't encounter. Thanks to @nataliejschultz we have the ability to work closely with someone running a Samsung device with Android Pie or higher (the most likely to have issues).

Initial Data: After setting up the app, Natalie took a few trips, and encountered weird behavior. She saw WSOD while trying to check the app on the trip (we're thinking this was just connectivity related) but also "teleported" as a part of the trip. The clip of screenshot below shows the location jumping West (note she did not go to the place marked as trip end at all). Accurate trip on left, Samsung trip on right.

Not shown, to keep personal data private, but she was also able to show me a comparison of the same trip on three phones (production, testing on Samsung, testing on iPhone) and the three were roughly equal in terms of time and distance, but varied in accuracy - some, including Samsung, had more of the "straight line" behavior than others.

Changes made: After seeing the weird behavior, we walked through the directions on "don't kill my app" together and turned off "auto optimize" in her battery settings as well as "put unused apps to sleep" in an advanced page of battery settings.

Next Steps: She is going to continue carrying the phone and report back on if the performance changes at all.

Abby-Wheelis commented 1 year ago

turned off "auto optimize" in her battery settings as well as "put unused apps to sleep" in an advanced page of battery settings. -- me

Are these settings per-app or do they affect the entire phone? - @JGreenlee

yes, if I'm remembering correctly, @nataliejschultz can correct me if I'm wrong. We were following the instructions in the don't kill my app site about Samsung > Android 11 to check on things. If it is for the whole phone, is that bad? I haven't researched into the app store approvals, but I can see where prompting to alter whole-phone settings might get in the way of that.

Abby-Wheelis commented 1 year ago

relpy from @nataliejschultz

A potential happy medium might be to non-invasively suggest that users can turn off certain phone settings for app optimization without forcing them to? I only suggest it because of the qualitative difference between the accuracy of my iPhone trips vs my Samsung trip (a separate trip that didn’t see the big jump in the screenshots above).

@Abby-Wheelis the settings we changed today were for the whole phone, because all of the OpenPATH app settings were already set to what was suggested by the setup prompts :)

Abby-Wheelis commented 1 year ago

And the comment from @shankari that led us back over to this thread!

@nataliejschultz my concern, based on prior experience with not requiring permissions, is that users are just going to blow past all suggestions with "yadda-yadda-yadda" and then complain that "the app is not working". This is not a new discovery, as @Abby-Wheelis saw, we have had special cases for Samsung for years.

Here's the prior issue where we discussed this for context: https://github.com/e-mission/e-mission-docs/issues/535

As you can see from dontkillmyapp, it is not just Samsung, but also Huawei, TCL, etc

From a logistical perspective, we should:

close one of these (probably this one),
copy the discussion to the other one,
brainstorm solutions, potentially building on what is already there
my high-level thoughts are in the issue (tl;dr: I think we should expand the app status beyond permissions similar to the dontkillmyapp android app but focused on the functionality we use

Abby-Wheelis commented 1 year ago

I forgot one of @shankari's comments from the other thread:

I don't think that this is necessarily related to Samsung phones - you see the "big jump" issue on your test iPhone as well. I think that the Samsung phone permission issue should reflect as seeing no points not bad points. But Samsung is weird so anything is possible.

If this fixes it, do we really want to tell people to turn these settings off manually? Note that:

the instructions are different for different phone models.
there is no way for us to get them to the screen automatically
and no way for us to check if they have actually done so.

Abby-Wheelis commented 1 year ago

you see the "big jump" issue on your test iPhone as well

I did want to chime in here and say that I've never seen a "big jump" on my test iPhone, but do frequently see "straight-line" trips, where only the start and the end of the trip and maybe one other point are marked on the map, and connected with straight lines. This does result in some funky route maps (crossing over tops of mountains), but I've never seen teleportation.

If this fixes it, do we really want to tell people to turn these settings off manually?

Given all the issues, it makes sense that it wouldn't be really worth it to ask them to make those changes. I agree that we should keep brainstorming solutions to best support the functionality of our app.

shankari commented 1 year ago

@Abby-Wheelis the settings we changed today were for the whole phone, because all of the OpenPATH app settings were already set to what was suggested by the setup prompts :)

This is not necessarily true. Note that android, unlike iOS, is a fragmented ecosystem. Samsung has introduced settings that are not part of Google's android (e.g. are not present on Pixel phones) and has no API specific method to manipulate them. e.g. "Lists of Sleeping apps, Unused apps, Deep sleeping apps, Never sleeping apps" are all app-specific.

Note (in the history here) that there are third party libraries that have tried to bridge the gap by reverse engineering the non-stock android setting page names and trying to pop them up. Given my limited resources and the sketchiness of the libraries, I have so far resisted going down that path of whack-a-mole but am open to revisiting creating something similar ourselves if there is interest.

Abby-Wheelis commented 1 year ago

@nataliejschultz sent me information about another trip that she took yesterday, as you can see below, the Samsung (right) is not tracking all that well, even with the battery changes we made. Left is iPhone. NatalieIphone

shankari commented 1 year ago

I think next steps are:

changing other settings (unused apps...)
another couple of recent samsung phones (just in case this has wonky hardware)
keeping the app in the foreground during the trip

Abby-Wheelis commented 1 year ago

@nataliejschultz, could you try investigating other settings (like unused apps), and keeping the app in the foreground during a trip and see if that decreases the wonkiness at all?

In terms of the hardware, would trying to use another location based app, like google maps or Gaia (free gps app with a tracking feature, often used for things like hiking) be another way to test for wonky hardware?

Abby-Wheelis commented 1 year ago

I talked to @nataliejschultz again today, and she had a few observations about her Samsung phone:

really struggles with walking trips (adding circles to the walk, even with app open in the foreground)
some walking trips zig zagging wildly
Samsung seems to do well with car trips, knowing what roads were actually used
also some issues with shorter trips

Maybe it's a fitness permission thing? But those are enabled. Checked on a way to improve location accuracy by turning on wifi and bluetooth scanning, but those were already on.

Google maps shows the same current location between the Samsung and her personal phone, which we checked in an attempt to see if the gps on that particular phone might be off, but it seems fine at least sometimes.

@shankari any thoughts about it mostly being walking that's so off? One theory we have is the dense urban area she's tracking in could make it harder for location services to keep up.

nataliejschultz commented 1 year ago

Google maps shows the same current location between the Samsung and her personal phone, which we checked in an attempt to see if the gps on that particular phone might be off, but it seems fine at least sometimes.

Yesterday I took a walk with google maps open on both my personal phone and the Samsung phone. I noticed that the tracking on my personal phone was (unsurprisingly) better. However, depending on how OpenPATH draws the trip, the difference should have been negligible. I would estimate that, when the location bubble would deviate, it was approximately 10 feet off from the location bubble on my personal phone. It would appear to be across the street or ahead of me, but never behind me.

Note: I was walking in an area away from the high-rise buildings downtown. Next, I will try re-taking the path that caused a zig-zag walking trip on OpenPATH (ie my commute to campus).

shankari commented 1 year ago

@nataliejschultz

However, depending on how OpenPATH draws the trip, the difference should have been negligible.

How did openpath draw the trip?

nataliejschultz commented 1 year ago

How did openpath draw the trip?

I was so focused on the google maps bubble, I didn't even think of looking at the actual trips. How silly! I've attached photos of the trip from both phones.

It looks like I was definitely wrong about the negligibility of the drifting location bubble. The iPhone trip is more accurate, although the boomerang at the end of the trip isn't accurate.

Still, it's much better than the highly inaccurate zig-zag trip (mentioned previously), which I've attached a photo of as well.

Samsung Walking Trip 10/3	iPhone Walking Trip 10/3

ZigZag Walking Trip on Samsung	ZigZag Walking Trip on iPhone (Much more accurate, but still showing boomerang effect)

niccolopaganini commented 1 year ago

Manufacturer: Samsung Model: ZFlip3 App Version: 1.5.2 (correct me if I am wrong) Android Version: 13

I have classes and a couple of club events today so a lot of walking today :smiley_cat:. I will keep an eye out and for a few trips, let it run in the foreground and then report my findings here tonight. I can also take it to trails for my bike rides tomorrow and report my findings later.

Abby-Wheelis commented 1 year ago

@niccolopaganini Do you have any updates about what you found while walking or biking over the last few days? Are the trips recorded accurately on the Samsung phone you have?

niccolopaganini commented 1 year ago

**This is the trip I took on 10/5/2023. I noticed it seems to have a similar problem to @nataliejschultz. Photo for reference:

I have drawn in red the path I have taken for you to look at. I took a Google Pixel 4a with me running Android 13 to compare the trips.

Here are the screenshots for comparison (Samsung followed by Pixel):

I have taken more trips later and can update the issue if you need more evidence.

Abby-Wheelis commented 1 year ago

It looks like you're having similar behavior to Natalie, where shorter/walking trips are not particularly accurate, but longer trips are mostly fine. @niccolopaganini would you say that summary is accurate?

If we identify this as a pattern consistent across phones, maybe we can have a better place to start researching a solution in terms of permissions needed.

shankari commented 1 year ago

@nataliejschultz @niccolopaganini can you edit your comments to put the images side by side so they are more visible? I've edited https://github.com/e-mission/e-mission-docs/issues/535#issuecomment-1747496757 as an example

Finally, I don't think that any of these trips are actually that inaccurate. There is some noise, but a good map matching algorithm will fix it. What I would be really worried about is https://github.com/e-mission/e-mission-docs/issues/535#issuecomment-1694498426 where the route is completely off by multiple blocks.

I'm planning to meet @nataliejschultz tomorrow so that we can walk around with an array of phones to see if we can re-create the completely off error.

niccolopaganini commented 1 year ago

It looks like you're having similar behavior to Natalie, where shorter/walking trips are not particularly accurate, but longer trips are mostly fine. @niccolopaganini would you say that summary is accurate?

If we identify this as a pattern consistent across phones, maybe we can have a better place to start researching a solution in terms of permissions needed.

I guess your interpretation is accurate (when trips look something like this:	However, what I am concerned about is when trips look something like this:

I thought of taking more bike trips but I was physically unable to. Given that @shankari & @nataliejschultz will be logging some trips today, we might be able to have a much clearer picture.

(Screenshots taken from Strava)

nataliejschultz commented 1 year ago

I'm planning to meet @nataliejschultz tomorrow so that we can walk around with an array of phones to see if we can re-create the completely off error.

It was so fun to meet Shankari and try out different aspects of the app in person! I walked to Union station, then walked with Shankari down 17th and turned right on Glenarm. At about the halfway mark, we made a it point to keep the app open on our test phones with the screens unlocked. Here are my results:

Samsung Galaxy A03s, Android version 13, OpenPATH 1.5.3

To Union Station	Union Station to Midway	Midway Onward (App Open, Screen Unlocked)

iPhone 13 Pro, iOS 16.6.1, OpenPATH 1.5.1 (Prod)

To Union Station and Midway	Midway Onward (App Open, Screen Unlocked)

iPhone 8 Plus, iOS 15.5, OpenPATH 1.5.3

To Union Station and Midway	Midway to End

Note: I did not keep the iPhone 8 screen open at any point, although this was the most accurate trip. Going under a bridge at the very end of the trip did not seem to alter the tracking in any way!

e-mission / e-mission-docs