Track recording stops by itself and resumes when screen is turned on despite proper settings

speleoluc commented 2 years ago

Description

While paddling with phone on kayak deck, the GPX track recording sometimes stops, but turning the screen on resumes recording. This is despite Android's Power Save mode being off and Standalone recording enabled in OSMAnd+.

OSMAnd+ was the only open window. Interestingly, with track recording on, most of the time (but now always), just tapping the phone's Home button displays OSMAnd+ without unlocking. When I'm done checking the map, I just turn off the screen with the Power button. At other times, still with track recording on, I get the locked desktop screen and must unlock the phone to get to OSMAnd+.

In the attached GPX (TXT appended for GitHub) track, 30-s interval recording stopped at 2022-08-13T16:20:06Z and resumed 18' later (2022-08-13T16:38:55Z) when I turned on the screen.

2022-08-13_11-20_Sat.gpx.txt:

A friend with a different phone has the same gaps in some of her tracks.

What else must I configure to get whole track all the time?

How to reproduce?

Launch track recording, move and wait.

Your Environment

OsmAnd Version: 4.2.7 Android/iOS version: 8.0.0 Device model: Samsung Galaxy A5 (2017)

Maps used (online or offline):
If you have an issue related to offline maps, tell us the exact name of the map file where the issue occurs and its edition date.

sonora commented 2 years ago

Have you tested my hints published here https://osmand.net/docs/user/troubleshooting/track-recording-issues/, in particular my "Tested Settings"?

speleoluc commented 2 years ago

Yes, I read and followed those instructions.

I'm under Android 8. In Parameters/Maintenance/Battery/Unmanaged Apps, OSMAnd+ is there. The description says that these apps can use as much battery as they want, and power management will do nothing to stop them. Power Saving mode is off too.

sonora commented 2 years ago

I just noticed you already had opened #14985, and I was under the impression the culprit was that you had not allowed standalone logging in the settings, but it seems it was not?

Unfortunately I have no Android 8 device any more for testing and cannot say how exactly they perform with all code changes required by new Google policies lately. If the track recording resumes at points where you picked up the device and turned on the scteen, this would indicate some sort of background behavior issue, I would say. You could probably test by leaving the app in tbe foreground with the screen on at low brightness, to confirm. Do you have a second device with a more recent Android version to test?

sonora commented 2 years ago

@vshcherb I had streamlined our article about it some time ago first here content/osmand/troubleshooting/track-recording-issues.md and then here main/docs/user/troubleshooting/track-recording-issues.md, but neither shows up on our web site at https://osmand.net/docs/user/troubleshooting/track-recording-issues? Which file is the correct one to edit, and what build to trigger afterwards?

vshcherb commented 2 years ago

@sonora this server updated automatically - https://maptile.osmand.net/docs/user/troubleshooting/track-recording-issues/, for main one you could trigger (https://osmand.net/docs/user/troubleshooting/track-recording-issues/) i.e. you have permission but it's a manual thing. Please use automatic one and once a meanwhile we will update main one 1-2 days .

speleoluc commented 2 years ago

I just noticed you already had opened #14985, and I was under the impression the culprit was that you had not allowed standalone logging in the settings, but it seems it was not?

My earlier case was closed without an explanation.

Unfortunately I have no Android 8 device any more for testing and cannot say how exactly they perform with all code changes required by new Google policies lately. If the track recording resumes at points where you picked up the device and turned on the scteen, this would indicate some sort of background behavior issue, I would say. You could probably test by leaving the app in tbe foreground with the screen on at low brightness, to confirm. Do you have a second device with a more recent Android version to test?

I do have an Android 11 tablet. So far, I've had similar recording problems. I'll be receiving today a new waterproof case to bring it paddling.

speleoluc commented 2 years ago

The Android 11 tablet seems better at keeping OSMAnd+ alive. My two weekend paddles were flawlessly recorded.

Possible hint: Google Maps on my Android 8 phone sometimes alerts me GPS signal gets lost while driving. What would happen then with OSMAnd+ track recording? Would it store unknown position records or just stop recording until the screen is turned on again?

sonora commented 2 years ago

OsmAnd would only record a point if a position is known at the time. Hence if e.g. there is a hardware issue or you do not have GPS reception for any reason, that would be conssistent with the recording exhibiting a "gap".

speleoluc commented 2 years ago

If the phone loses GPS data while OSMAnd+ is recording, does recording resume when GPS data resumes? Or does it only resume when I turn on the device screen? This could be the cause of the problem. A hardware failure and a lack of fault tolerance on the software side.

sonora commented 2 years ago

Hard to say, it likely depends on the nature of the hardware issue. What I can say is that for a mere GPS reception gap, like when I climb through a slot canyon, OsmAnd automatically records each due point if a position fix is available at the time, regardless of history: To the effect that both interim singular (in the middle of the "gap") points with occasional reception are recorded ok, as well as the reception resumes ok after leaving the canyon, all with the phone untouched in my pocket.

But I can envision there could possibly be a phone behavior, intentional or hardware issue, disabling GPS entirely until the screen is turned on again, which would fit your observation, and with no chance for OsmAnd to mitigate. (I have no such device, though).

Have you tested the device over land? Because possibly that particular GPS chipset has issues with reflections if close to the water table?

speleoluc commented 2 years ago

I paddled most days last week. On the first four, I ended up with gaps in the OSMAnd+ recordings (despite an open sky and the tablet left in a waterproof ZipLock-like pouch on deck). At times, turning the tablet screen on, I noticed there was no GPS position available and the app was stuck on searching satellites. I had to turn the Android GPS off and on to get a position. Then, for the last two days, I simultaneously launched track recording by OSMAnd+ and Gaia GPS. This time, in just one trip, I only get a (less than) one-minute gap that fixed itself automatically - the only sign of this was a stack of multiple alerts reporting the lack of a GPS position by OSMAnd+.

BTW, when reporting on a track point, OSMAnd+ should display a timestamp with seconds, given the recording frequency can be as high as 1 Hz...

sonora commented 2 years ago

But still only the Android 8 device that's affected, right?

It's a wild shot, but it seems that although you have "Include heading" enabled for your recordings, the track you originally posted contains only 32 times a heading (out of 576 recorded points). Does turning off the "Include heading" switch make any difference for your issue?

speleoluc commented 2 years ago

This "Include heading" option was only enabled on my phone, but already disabled on the tablet that I've been using all week. Here's the gapped GPX, although I don't see any gap in the data, only in the map view when we stopped for lunch on a beach. Enabling the "Join gaps" option makes the lunchtime gap disappear.

BTW, is there a way to tap on a specific point of the displayed track to know its exact time?

So for the moment, my best bet seems to be to start the track recording in Gaia GPS to keep the Android GPS alive for OSMAnd+...

2022-09-11_Monroe.gpx.txt

sonora commented 2 years ago

Took a brief look and that new track looks fine to me:

It covers a time span of roughly 278 minutes.
There is a 37 minute gap at 2022-09-11T17:56:43Z, I assume this was your lunch break and you intentionally (or via motion filter) interrupted the recording (?)
Hence the track covers 241 minutes
It contains 2683 track points, roughly one every 5.38 seconds, quite consistent assuming you selected a sampling rate of 5 seconds? (I see no periods of missing points, only occasional intervals of 6 instead of 5 seconds, probably due system timing behavior of acquiring the point and writing it.)

I am confused now which track is from what device under what Android version, and which track was taken with another GPS recording app being active an the same device in parallel. What bogs me is whether it really makes a difference or not having a separate app active, because that would indicate there is a better way to do something in our code... So I am interested in getting clear evidence if is the case.

PS: Other than tapping any of the speed analysis graphs (see 2 screenshots in #15284), then swiping your finger, I am not aware we have a good mechanism of identifying points on the track by tapping them. (Used to have it some years ago that track points were tappable, but that disappeared by code refactoring.)

PS2: Regarding enabling "Join segments": Your track contains 2 segments (denoted by <trkseg> if viewed in a text editor). If you scroll up the analysis graphs on the "Track" tab, you will first see the plot for the combined data, then below the analysis graphs for each segment separate. Without "Join segments", the combined graph shows the data for each segment plotted one behind the other but without relating them, hence any possible gap between segments does not become visible. With "Join segments", all data from the different segments seems to be analyzed as if it belongs to one combined segment, i.e. formerly hidden gaps between segments will now appear in the graph as explicit gaps within one combined segment. Use you first track to try this out, because your second track only contains a gap between segments in the time domain (lunch break), but you resumed at the same location, hence no gap vs. distance).

speleoluc commented 2 years ago

Previous tracks were from my phone. Only this last one was from my tablet, for which I recently purchased a waterproof case.

I never intentionally pause the recording at lunchtime or at any other stop, I always leave everything running to document this gap problem.

I attach a screenshot of my recording settings (mostly defaults).

I also attach a gapped track from same tablet (Samsung S5e) taken earlier last week, without Gaia running concurrently. I can't explain the first 2 km gap after Heafy island. The second gap was a visit to a dam, when the kayak stayed idle for a while, then another gap at lunchtime, without ever pausing the recording. Further south, a final 1 km gap before getting to the road.

2022-09-08_Cabonga.gpx.txt

sonora commented 2 years ago

You use an accuracy filter for recording, but under clear skies that is an unlikely culprit here.

But let me verify: On the Android 11 device, have you made sure that both the 'Power saving mode' and the 'Adaptive power saving' setting are turned to OFF? Particularly the latter one is often overlooked and has caused me headaches before discovering it...it is rather stashed away on a different screen, but if enabled can actually kill processes by changing the power mode even if power saving is generally turned off.

speleoluc commented 2 years ago

I'll turn off that accuracy filter.

Is the "Auto-split" option causing the start/stop icons on the map when the tablet remains immobile for a while?

In the Control Panel, the Power Save icon is off. I didn't know about the Adaptative Power Save, but I see it's off. Only the Daily Automatic Optimization was on, and I turned it off for good measure.

I will report back when I go paddling again.

sonora commented 2 years ago

"Daily Automatic Optimization" has actually never caused an issue for me. I have documented the exact settings I have successfully used for several generations of devices (mostly Samsung) here: https://osmand.net/docs/user/troubleshooting/track-recording-issues, in section "Tested Power Settings for....". Please check closely. The exact location of some of these settings varies with Android version and even with device manufacturer. I cannot guarantee that these will work in any case, but can report that I have not experienced a single gap in any of my recordings (I use OsmAnd professionally for scientific purposes and expeditions) I would attribute to any power savings issue. But I admit that researching and re-testing these settings on new devices and new OS settings has been a pain.

Regarding OsmAnd's Auto-spliit setting: A gpx track file is capable of grouping recorded track points in 'segments', you can have one or several segments in a file. (The corresponding setting "Join segments" instructs OsmAnd for the track analysis graph to either respect or ignore if a track is separated in several segments. BTW: In an existing file you can simply (using a text editor) insert or remove segment boundaries by adding <trkseg><|trkseg> lines where needed.

Whether OsmAnd shows or hides Start/Finish icons in tracks (including for segments) is a setting under the Track appearance options (Show start and finish icons).

OsmAnd's"Auto-split" option governs if a recording gap introduced by any reason also introduces a new segment start (which later helps making these gaps visible), or if recording after a gap simply resumes in the same segment. I just checked and I have Auto-split on, it seems our default in OsmAnd.

speleoluc commented 2 years ago

Yesterday I went paddling and recorded the trip. Power saving off, adaptative power saving off too. I deselected the Minimum GPS accuracy setting in OSMAnd+ but I didn't launch Gaia GPS concurrently. At one point, I turned on the screen to know my speed under sail-only P9150890

the tablet is in the waterproof pouch sitting on my spray skirt in the forefront -, only to realize it wasn't displayed because there was no GPS position. I then turned off and on again the Android GPS and after a few seconds, recording was restored ( 2022-09-15_Varennes.gpx.txt ). All in all, I lost about 800 meters.

If the gaps in my tracks are due to a lost GPS signal (!), could the software at least warn me orally, like it would do while navigating a route? Or better, restart the GPS itself?

sonora commented 2 years ago

Thanks for the gorgeous picture! ;)

You will receive oral warnings "GPS signal lost" and "GPS signal recovered" if you have a navigation running in parallel, I have just tested that: Pick an arbitrary point on the map, start a voice navigation to it, and ignore the initial oral warning that you are off the route when you paddle elsewhere. But the navigation will remain active, and with it the oral GPS warnings we have implemented there.

Maybe it is a reasonable test a recording with a navigation in parallel, but am not sure this gives any more insight.

Regarding the new track: It has mostly single digit hdop values, meaning it has rather reasonable accuracy. Immediately before the gap it shows hdop=3.9, after 4.2. I see no indication at all that the GPS reception would have failed so suddenly and so completely due to signal strength issues, while there is no other gap anywhere else in that recording. I have to say I am a bit at loss here what to test next, and still have no idea what if anything could be improved about it in the code.

When you turned on the screen, OsmAnd was running, but its GPS widget reported 0/0 satellites, right? (Or at least the system notification in the top bar was visible to indicate the active recording, displaying the distance already logged?) I am just trying to rule out that you have some other memory-hungry app running which at times causes Android to close other processes.... But I do not think Android would close active foreground services.

speleoluc commented 2 years ago

I put the picture to show the tablet had a good sky view, forgetting about the hdop stored in the GPX file ;)

Yes, when I turned on the screen during the gap, the GPS said it was searching for satellites.

I always close all apps when paddling, if only to avoid confusion when handling the screen with droplets on the pouch. That being said, Android doesn't really stop apps, it mostly just closes windows. I would have to reboot it and just start OSMAnd+ to get a cleaner state - I'll do that next time.

I'll try the navigation trick to get a voice warning about lost GPS reception.

sonora commented 2 years ago

Yes, let's find out if OsmAnd actually reports a loss of GPS reception in these cases, take it from here.

There is also a GPS widget you can configure to be visible your profile's map view, using "Menu/Configure screen". It will show the number of satellites visible and in use on the map screen.

At the same time, can I ask if other users see this same phenomenon, stating the circumstances and devices. But before reporting here, can you please make sure you meticulously follow the Tested Power Settings I have outlined here https://osmand.net/docs/user/troubleshooting/track-recording-issues/ in sub-section '"Tested Power Settings for Android...'', so that we don't dilute this particular issue by any power saving issues which those documented Android settings already mitigate.

speleoluc commented 2 years ago

I already have the GPS widget visible. When there is no reception, it will report something like "Searching GPS 0/39" or 0/0.

A friend with an Android 8 Samsung S7 also experiences track gaps. I'll see her in a club outing this weekend, and we'll do another try with all options set per your document.

On my Android 11 tablet:

All my Power Save options are disabled.
Adaptative battery is off.
Unused apps put to sleep was on, but how can this apply to OSMAnd+ while recording? This setting is valuable since Android doesn't stop apps when closing its window. Gaia GPS appears in the list of apps never put to sleep, but trying to add OSMAnd+, I notice it doesn't show up in the list of possible apps - strange.
I don't find Optimize settings.
In Parameters, I see applications can be battery-optimized, and this setting is applied to almost all apps; I just removed OSMAnd+ from this list (BTW, I see Gaia GPS is battery-optimized).

sonora commented 2 years ago

Regarding Adaptive Battery: Off should be fine, I even have it on.
Put unused apps to sleep: I believe I have observed crippled recordings with this on, hence in my "Tested Settings" I have this "off".
- But if I turn it on: OsmAnd does show up in my list to select exceptions! Maybe check this: Go to Android-Settings/Apps/OsmAnd/Battery: Is perhaps "Allow background activity" set to off by any chance? I only just now discover this, maybe it is new under Android 11.
Looks like Optimize settings was removed in Android 11, can't find it anymore either. I have updated the article.
Regarding Battery Optimization: I remember this used to do the trick on some earlier Android versions, so maybe your friend on Android 8 should try this out. Since at least Android 9 it did not solve the issue for my reliably, and due to that I stopped systematically testing that setting.

I had also become rather desperate about the ever changing Android Power saving strategies. And I can't even say how manufacturer-specific things are on top of this... I can only recommend to very strictly start from my tested settings as a basis, then take it from there.

speleoluc commented 2 years ago

But if I turn it on: OsmAnd does show up in my list to select exceptions! Maybe check this: Go to Android-Settings/Apps/OsmAnd/Battery: Is perhaps "Allow background activity" set to off by any chance? I only just now discover this, maybe it is new under Android 11.

For OSMAnd+, the "Allow background activity" is on but grayed out.

Those intertwined Android settings are getting more complicated with every version.

sonora commented 2 years ago

I have streamlined my article a bit primarily focusing on the 10 crucial settings to find, rather than their exact screen placement which varies wildly over different versions of Android. Hope it is easier to read and implement now!

Would be good to also find a set proven to work 100% on an Android 8 device, maybe we can achieve this here.

sonora commented 2 years ago

Regarding interrwined settings, I find the following:

Removing an app from "Optimize battery usage"

sets the "Allow background activity" button to on, and grays it out.
Removes that app from the selection list for "Never sleeping apps".

So once again, I think everybody's starting point should be exactly my 10 "Tested Power Settings" at https://osmand.net/docs/user/troubleshooting/track-recording-issues/, verify as a baseline if these work for you before experimenting.

speleoluc commented 2 years ago

After many trials, I must conclude the problem is Samsung's, not OSMAnd's. According to some developers forums, their GPS handling has gone worse with the latest updates. Every time OSMand+ stops recording, it is because it cannot get a position. Turning the GPS off and back on forces it to "reboot" and return a position. This does not only occur at slow (paddling) speed, I also see that at highway speed when, despite an open sky, Google Maps quickly alternates between a green direction panel and a gray alert of lost signal. I tried running the GPS Keeper app to help track recording, but it only keeps the GPS alive, without forcing it to reinitialize when it goes south.

pebogufi commented 2 years ago

@speleoluc What about contacting the developer of GPS Keeper to add a option for reinitialize.

speleoluc commented 2 years ago

I already did.

osmandapp / OsmAnd