Why would two identically configured devices record different numbers of data points?

[yobyot]

I'm hoping someone can give me a clue about something I see in Recorder that's puzzling me. The mystery (to me) is that one device records far more data points than a second, identically-configured device. Today, for example, both devices were in my possession and traveled to the same locations. Yet one device recorded 11 data points; the other device recorded one.

I have two devices configured identically. That is, they are the same physical device type (iPhone 15 Pro Max), on the same level of iOS, with the same level of the iOS OwnTracks app. Both are set to "significant" in OwnTracks. Both are set to allow OwnTracks to record precise location at all times.

In MQTT, both use the same server using websocket on 443 using TLS and record to similar topics, for example owntracks/iotClient/deviceX (where X is one iPhone or the other).

The iOS app MQTTLoupePro suggests that both devices are sending the same number and frequency of OwnTracks events. But for some reason, not all of one of the devices's events make it into a .rec file for one of the devices.

Is there some setting I missed that tells Recorder how many events per device per time period? Or maybe some kind of throttle?

I'd appreciate any clues.

[jpmens]

There is no setting which limits what the Recorder will accept, throttle or otherwise.

When you say the devices are identically configured, I assume they really are. Even so, you might wish to export setttings on both (or publish them to the Recorder — they’ll show up in STORAGEDIR/config*/*/*.json) and diff(1) them to be sure.

In order to debug this, what I would do is run two individual mosquitto_sub against your broker, one for. each of the topics the devices publish to. At the end of a time period you could then at least easily verify what arrived at the broker, irrespective of what an unknown to us iOS app says. :-)

I think this might give you much more insight as to what’s happening.

[jpmens]

Could you please also verify that the devices are configured with distinct MQTT clientIDs? You might be aware that a broker will kick off a client which is connected with a clientID of a new client which results it a kick-off fest. That might explain the phenomenon.

You'd also be able to see from your broker logs when devices constantly re-connect.

[yobyot]

Thanks, @jpmens.

I look into the diff when I have more time.

On the second point, this screen image should confirm that neither device is stepping on the other's events. They have separate topics in the owntracks tree. And you can see that the messages counts for "device 1" are about 30% greater than for "device 2" (device names have been obscured for privacy).

[jpmens]

The topic names don't matter in respect to the broker disconnecting a client; it's the clientID which is used during the connection.

In iOS OwnTracks this value is configurable in Settings, so it's worthwhile checking that these are distinct over two two devices.

Other than that, feel free to drop your configuration to us at support@owntracks.org if you want us to take a look.

[yobyot]

Thanks...I didn't realize you responded via email. I've emailed you there with more info.

Very much appreciate the help.

[jpmens]

Thanks for the well done screenshots which shows clientid is empty.

Our Booklet describes:

clientId client id to use for MQTT connect. Defaults to "user deviceId" (iOS,Android/string)

As your device names differ, this shouldn't matter.

[yobyot]

Thanks.

Just to be clear: the defaults do differentiate the inbound messages sufficiently to negate the possibility that one device's messages are overwriting the other's?

[jpmens]

Yes. There are two distinct things we need to look out for:

1. the clientIDs of MQTT clients (i.e. your OwnTracks devices) must differ so the broker doesn't kick them off
2. the <username>/<devicename> for the devices must differ as from that the topic is constructed (e.g. owntracks/jpmens/myphone) which is used by the Recorder for storage into the .rec files.

Both of these are the case as you've documented.

There are two next step in debugging the issue, IMO:

1. use mosquitto_sub over a few hours (as initially described above) to get a snapshot of what the broker is actually receiving. (This will have no influence over what the Recorder receives.
2. look through the Mosquitto logs to see whether devices are having connection issues.

There's one more thing you could do and that is to verify the device which reported very few locations had a particularly high number of lwt during that time:

$ grep -c lwt /var/spool/owntracks/recorder/store/rec/*/*/2024-02.rec

The numbers should be similar.

[yobyot]

Just an update.

I wrote a script that pulls data from Recorder using the API and writes it to .csv file only if the reported accuracy is less than or equal to 30m. That's why one I saw so many fewer records for one device compared to the other.

What I haven't been able to determine yet is why two identical iPhone models would report such wildly different accuracies when the devices are in the same location for the same amount of time.

Thanks as always for the suggestions.

[jpmens]

@yobyot how are your experiments progressing?

[yobyot]

@jpmens Thanks for checking in...and apologies for the lack of updates.

What I've been able to determine is that one of the two identical devices (they are both iPhone 15 Pro Max models, same memory, same version of iOS, same cellular carrier, same GPS/location permissions and settings) reports with less accuracy than the other.

Let's call the two iPhones device_a and device_b. I wrote a script that calls the API to retrieve records for both devices starting from a specific date like this:

device_a

http://10.69.0.58:8083/api/0/locations?from=2024-02-23&fields=isotst,tid,addr,acc,lat,lon,ghash&device=device_a&user=iotClient

device_b

http://10.69.0.58:8083/api/0/locations?from=2024-02-23&fields=isotst,tid,addr,acc,lat,lon,ghash&device=device_b&user=iotClient

The script then removes records for each device whose acc field is less than or equal to 30 meters before creating a .csv file as output.

The reason that I originally thought that device_b wasn't reporting was because many more of its records have acc fields that are greater than 30 meters. IOW, it's not as accurate as device_a.

I still don't understand this or why it's happening.

[ckrey]

@yobyot thanks for the analysis so far. We obviously cannot access your backend within the private address range 10….

There are 3 things you can check:

• on the map view the OwnTracks app displays the top right the current accuracy delivered. Do these number differ? This is the accuracy the map receives (kind of Move Mode) but it may give a hint
• are any other location apps installed on the “better” phone. OwnTracks does not access GPS or the other location provider directly but through iOS services. The services share locations determined by other apps. E.G. if you run a navigation app on the phone your iwntracks app in significant mode will receive more updates.
• do the two phones have the same mode number (Settings/General/Info) and do the use the same protective case?

[yobyot]

@ckrey Thanks!

We obviously cannot access your backend within the private address range 10….

Understand. OwnTracks is on a small Ubuntu 22.04 VM under Proxmox behind an HA Proxy, runs on TLS and requires password authentication. When I first took a look at MQTT, I decided it was way too open and most deployments I researched too expose for my comfort. I prefer not to directly expose anything to the public internet, as I am sure you can understand

on the map view the OwnTracks app displays the top right the current accuracy delivered

I just checked. Both devices are next to each other. The "better" one (device_a) shows 16 ft (I am in the USA). The "less good one" (device_b) shows 18 ft. Pretty close.

are any other location apps installed on the “better” phone.

Interesting. I have Google Maps on the better device but not on the less-good device. And OwnTracks on both devices is set to significant mode. Are you saying that if there were no GPS-collecting apps on an iOS device that OwnTracks would record no locations?

do the two phones have the same mode number (Settings/General/Info) and do the use the same protective case?

I don't see an Info selection in iOS 17.3. However, the carrier levels/modem firmware are the same on both devices:

• Carrier: T-Mobile 57.0
• Modem firmware: 1.40.1

There is one difference between the two devices, now that we are in the weeds looking: the "better" device has two eSIMs (that is, two active voice numbers). The "less good" device has only one. However, AFAIK, there's only one GPS. Moreover, the way Apple has implemented multi eSIM, only one of the two active numbers can be used for data.

[yobyot]

I'm hoping the attached, annotated composite screenshot may help explain what (I think) I am seeing.

The image is output from Postman, from which I made API calls like the one I described above. Times in the JSON output are shown in UTC; I am in UTC-5.

During a period in which the (arguably) identical devices were in the same location, as indicated by the ghash, the "better" device recorded two nicely accurate location records; the "less-good" device (bottom) recorded only one. In fact, as you can see from the count on the "less-good" device, this record is the only record recorded on this date so far.

What is causing the "less-good" device to record fewer records when set to the same frequency ("Signficiant") is what I am trying to determine.

[ckrey]

Interesting. I have Google Maps on the better device but not on the less-good device. And OwnTracks on both devices is set to significant mode. Are you saying that if there were no GPS-collecting apps on an iOS device that OwnTracks would record no locations?

No, I wrote "the phone your owntracks app in significant mode will receive more updates."

[ckrey]

During a period in which the (arguably) identical devices were in the same location, as indicated by the ghash, the "better" device recorded two nicely accurate location records; the "less-good" device (bottom) recorded only one. In fact, as you can see from the count on the "less-good" device, this record is the only record recorded on this date so far. What is causing the "less-good" device to record fewer records when set to the same frequency ("Signficiant") is what I am trying to determine.

As I pointed out, other apps on the phone using Location services may trigger additional update to OwnTracks. You may check which other apps have Location Authorisation in "Settings/Privacy/Location"

It may help to understand what is happening if you include the "t" (like trigger) field in the output. If it is not included this indicates iOS actually provided a new location to the app, other triggers are listed here

Using significant mode: "Apps can expect a notification as soon as the device moves 500 meters or more from its previous notification. It should not expect notifications more frequently than once every five minutes." via Apple

[yobyot]

Thanks, @ckrey

I guess I find it confusing that OwnTracks is triggered by other apps. And, when it receives a "direct" location record, the t field is blank.

It could be that the less-good device has fewer apps that trigger OwnTracks. But this seems like a flaw. Cross-app data sharing with no control seems like a problem on multiple levels. First, one might want multiple apps that trigger a location notification. Second, one might not want to share app A's triggers with app B's.

What am I missing?

[jpmens]

OwnTracks is not triggered by other apps. As @ckrey explained, GPS related apps cause iOS to "pollenate" OwnTracks with better / more frequent GPS information. Note that I explicitly say "iOS": we cannot do anything without the operating system giving us the data; neither its quality nor its frequency. Fact is when a navigation app is operating, OwnTracks gets more frequent data.

There's going to be only one way for you to get behind what's happening: ensure both devices become identical: same GPS apps, same settings in Privacy/Location, etc. And possibly even same number and class of SIM cards. (We've no idea what two SIMs do compared to one only.)

And if that doesn't provide the satisfaction you're after, there's but one organization who can help: Apple themselves. OwnTracks uses Apple programming interfaces, and that's all we can do.

[yobyot]

Thanks @jpmens.

I think I get it now -- but I would suggest that in the future you consider changing the t field so that it's not blank for an OwnTracks-generated record. If everything else going on in the device's GPS causes Recorder to note the source, it shouldn't require a test for absence (the not case) to determine which records are actually "pure" OwnTracks-generated record.

Thanks to you and @ckrey for all your help.