Track recording doesn't record in the background (Android 8, EMUI 8)

[vshcherb]

I noticed a bad bug in current release: during navigation Osmand very frequently shut down when running in background. I have installed Osmand+ 3.0.4 on Huawei P10 with Android 8.0.

The issue is that during navigation after some time Osmand close instead than keep running in background and consequently the navigation stops. I do not think it is related with strict battery optimization policy of Huawei phones, because it never happen before.

I started to notice this only with last version.

[Bambasti]

Tested another device I have. On a Motorola G (2nd gen.) on LineageOS 14.1 / Android 7.1.2 it's working in the background as well, like on Android 9. For me this issue seems to be about Android 8/8.1 only.

[gormo-osm]

The issue started for me with the upgrade of Android or EMUI to 8. Now both are at 8. Device is a P8 Lite 2017. GPX tracking stops ca. 5 Minutes after screen turnoff.

[st-anton]

Last week I tried the background tracking, and found, that it work again as it should do. I'm happy with it again as say thanks to the developers. Huawei Mate 9, with EMUI 8.0.0

[sonora]

Looks like this is now our master issue for this still very annoying problem. For me the current status is as follows: (Samsung XCover 4, Android 8.1, all device power management turned off, for good measure also exempted OsmAnd from all app optimizations. Location method is set to GPS only.).

• For GPX recording intervals up to and including 15s, for which we do not turn off GPS between recording points, track recording now works as expected, even while the screen is off.
• But for recording intervals of 30+ seconds, where we turn off GPS to conserve power, we still seem to encounter a location limitation while the screen is off: For e.g. 30 sec. I receive a recorded point only every 5 minutes.

I have found this https://hackernoon.com/android-location-tracking-with-a-service-80940218f561 but am aware that we are already using a foreground service and notification channel in OsmAnd, so this is likely no help. Also, with my above settings, I have NOT seen OsmAnd being killed in the background, so that's not the issue any more.

From what I see, we now have a clear distinction between when it works and when not, I observe the issue only in the cases where we shut down GPS between measurements. So it should be fixable.

This is one of the major bugs we now have in OsmAnd, it definitely deserves to be in a milestone with adequate priority!

[sonora]

Looking at our code, it seems that we use AlarmManager.setRepeating to trigger recording a point for recording intervals >=30sec, see https://github.com/osmandapp/Osmand/blob/fad43918f1665669aacd1d0c1683fd7b5fdf05d5/OsmAnd/src/net/osmand/plus/NavigationService.java#L162

As per Android documentation:

Note: Beginning with API 19 (Build.VERSION_CODES.KITKAT) alarm delivery is inexact: the OS will shift alarms in order to minimize wakeups and battery use. There are new APIs to support applications which need strict delivery guarantees; see setWindow(int, long, long, android.app.PendingIntent) and setExact(int, long, android.app.PendingIntent). Applications whose targetSdkVersion is earlier than API 19 will continue to see the previous behavior in which all alarms are delivered exactly when requested.

and

Note: as of API 19, all repeating alarms are inexact. If your application needs precise delivery times then it must use one-time exact alarms, rescheduling each time as described above. Legacy applications whose targetSdkVersion is earlier than API 19 will continue to have all of their alarms, including repeating alarms, treated as exact.

I guess we have to use setExact and always re-set for API 19 and up??

[vshcherb]

Yes we need to be more than API 19 and try the approaches you mentionned. I will put it in 3.4 release so we don't forget about it.

[sonora]

More reading on the subject: https://stackoverflow.com/questions/33110246/setexactandallowwhileidle-is-not-exact-as-of-developer-reference. Would have thought we simply replace our 2 occurrences of setRepeating by setExactAndAllowWhileIdle, and re-schedule in onReceive, but it seems unclear if this works reliably...

[sonora]

I have created a pull request laying a foundation by using the correct setExact / setExactAndAllowWhileIdle for higher Build levels.

It does not fix all issues for all devices, but I guess we may have to implement this as a step 1 in any case. Possible step 2 as outlined here: https://wangjingke.com/2016/09/23/Multiple-ways-to-schedule-repeated-tasks-in-android

[sonora]

My test results after this commit:

• [x] (1) Screen is on, OsmAnd is the foreground app: On all devices I have access to (Android 4.1 - 8.1), logging now works as expected, both for short-interval (<=15sec) and long-interval (>=30sec) logging intervals.
  • [x] (2) Screen is on, OsmAnd is not the foreground app The same is true,, see (1)
  • [x] (3) Screen is off, up to and including KitKat versions: All logging also works as expected
  • [ ] (4) Screen is off, Android 6+ devices: This is still the major issue: On most devices the points recorded soon drop to a 3 minute, 5 minute, or even 10 minute interval, depending on device.
  • [ ] (5) Interim missing points: As a side note (but no new observation), in addition, particularly for device-off cases, some scheduled measurement points are missing in the resulting GPX file. Maybe this is due to a GPS fix not being established in time prior to the device stopping to try.
  • [ ] (6) Screen is off, Android Notification bar not correctly updated on some devices: This is a side effect, at least on some Android 8+ devices: While the device has in fact actually recorded many points (albeit e.g. at myaybe 5 minute intervals), the Android notification may not reflect this correctly when the screen is turned back on. The display in the notification may be far behind indicating the actual track which has been written. Keep this effect in mind when testing, inspect the actual GPX file, do not rely on the notification.

The major functional issue is clearly (4), i.e. the device not repeatedly waking up at exact times in doze mode for new Android versions, still not sure how to solve ...

PS: The wakeup issues seem to only exist while devices are not being charged.

[sonora]

The only (I hope temporary) workaround I can think of for now is that for logging

• for Build.VERSION.SDK_INT >= 19 with logging intervals 30 sec <= x < 5 min

we also resort to using serviceOffInterval = 0, i.e. keep the GPS on continuously (as we do today for intervals <= 15 sec). This would at least restore the long-interval logging functionality also for new devices, however at the cost of higher battery drain.

If we want to go that route: We have hard-coded the 30 sec limit in AppInitializer.java, OsmAndMonitoring.java, NavigationService.java, and perhaps elsewhere. Can provide a pull request for verification if desired.

[sonora]

I have created pull request "Issue 5632 Workaround". Commenting out the first if block in navigationServiceGpsInterval() will re-establish our old (and desired) default behavior.

[sonora]

First test results on one device (Android 4.4.2):

• The pull request works as expected
• Battery use is about 5% per hour on that device, not nearly as bad as I would have thought
• The issue of occasionally missing points (number (5) further up) has entirely disappeared, and the accuracy of recorded tracks has significantly improved, due to the fact that a GPS fix is kept if possible.

From what I see now we should go with this solution at this point.

EDIT: Now tested also on an Android 6.0.1 device, that one was more like 2-3% drain per hour, so really not bad... EDIT2: Tested 30sec logging with Samsung A5: 2% battery use per hour (26mAh)

[vshcherb]

Great work! @sonora

[sonora]

As per Android documentation, repeated AlarmManager reception has been severely limited starting from API>=19.

But let me for now apply this "keep GPS running instead of using AlarmManager" workaround to only Android 5 and up: Many Android 4.4 devices do still not severely limit their AlarmManager behavior, hence for these keeping AlarmManager may be ok and avoids the increased battery use (about 1.2% vs. 5% per hour of background logging for a Samsung S4mini).

Please report if there are any issues with Android 4.4 devices justifying to also include them in this fix (unfortunately things may be manufacturer-specific).

[sonora]

To consolidate our documentation on findings and strategies regarding background behavior, I have created a brief Tech Article on Background Track Recording - Strategies and Known Issues here: https://osmand.net/help-online/technical-articles#BackgroundRecording.

[e-pirate]

Heh, I'm here from #7083 . GPX recording just simply stops on my BlackView BV9500 if a lock the phone with OsmAnd+ not in foreground making the app almost useless.

[scaidermern]

Track recording (with interval set to continuous) stopped on my device (Xiaomi A1) after about 1.5 hours with OsmAnd being in the foreground but display off.

OsmAnd 3.4.0 on Android 8.1 (LineageOS) . Roughly the same behavior as in the previous OsmAnd version.

[sonora]

@e-pirate Please try OsmAnd 3.4, from what I can see on my devices that issue should be fixed now.

@scaidermern That one sounds like the Android 8.x Power management killed OsmAnd after some time. Have you tried white-listing OsmAnd in the Android Power settings? (Or, for the sake of testing, just disable Android's Power saving entirely on the device?)

[scaidermern]

@scaidermern That one sounds like the Android 8.x Power management killed OsmAnd after some time. Have you tried white-listing OsmAnd in the Android Power settings? (Or, for the sake of testing, just disable Android's Power saving entirely on the device?)

So far I didn't white-list OsmAnd. I will do this, thanks.

[heitorPB]

With OSMand on foreground and Trip Recording plugin recording a track, if I lock the screen OSMand stops recording the track. But If I unlock the screen again, OSMand adds another point to the track. Maybe the problem is I/O when screen is locked? It doesn't look like the system lost the GNSS fix.

Osmand 3.3.8, XIAOMI Redmi Note 4x, MIUI Global 10.2 with Android 7.0.

[sonora]

I have summarized here: https://osmand.net/help-online/faq#trip_recording_stops

[topas-rec]

Maybe @sonora you can add the following to the Osmand help pages:

"Lock the application in the application switcher (list active applications - locate running Osmand app - find the lock command and lock the app)"

I experienced that the Osmand app got stopped in background during navigation on highway and also during track recording on bike on a OnePlus 5t with unmodified vendor OS which runs Android 9. I then tried the suggestions in the FAQ without success. Since a few rides (only track recording on bike, rare navigation in car) the issue seems solved by using the lock option if an app.

Perhaps one who has this issue can also try this suggestion.

[ioctl-user]

Please see patch prototype here: https://github.com/osmandapp/Osmand/issues/9097

It fix big-timeout-wake-up-problem.