traccar / traccar

Traccar GPS Tracking System
https://www.traccar.org
Apache License 2.0
5.16k stars 2.56k forks source link

Data not displaying in Traccar despite correct logs (?!) #5303

Closed kusmierz closed 5 months ago

kusmierz commented 5 months ago

Describe the bug Although the device sends data correctly (as confirmed by the server logs), sometimes during the trip, some part of data (usually 1-2 minutes) does not appear in the Traccar app or get stored in the database. Server logs show that data is being received but hasn't been processed.

To Reproduce Steps to reproduce the behavior: The issue occurs frequently, almost once during each device tracking route (but only for 1-2 minutes pause), but a specific trigger causing the malfunction has not been identified. It is not possible to provide deterministic steps to reproduce the problem due to its intermittent nature.

Expected behavior When a device sends data to the Traccar server, the data should not only appear in the server logs but should also be visible in the Traccar app and stored in the database.

Screenshots image fun fact, speed is not the same as in the logs, does Traccar calculate it differently?

Additional context Partially obfuscated logs (at least hopefully)

2024-04-19 11:12:55  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031112541904245005028600019571601e029202ffffffffff041d000000000001cc00ff0000d600990000000024
2024-04-19 11:12:55  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:12:54, lat: 50.08381, lon: 19.95267, speed: 29.0, course: 202.0
2024-04-19 11:13:10  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031113091904245005003100019571455e000203ffffffffff0416000000000001cc00ff0000d7009b0000000025
2024-04-19 11:13:10  WARN: Unknown device - 2470000017 (37.???.???.???)
2024-04-19 11:13:25  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031113241904245005001600019571468e000203ffffffffff0418000000000001cc00ff0000dc009a0000000026
2024-04-19 11:13:25  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:13:24, lat: 50.08336, lon: 19.95245, course: 203.0
2024-04-19 11:13:40  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031113391904245005000500019571475e000203ffffffffff0418000000000001cc00ff0000df009b0000000027
2024-04-19 11:13:40  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:13:39, lat: 50.08334, lon: 19.95246, course: 203.0
2024-04-19 11:13:45  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031113451904245004996700019571418e010229ffffffffff041c000000000001cc00ff0000e400980000000028
2024-04-19 11:13:45  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:13:45, lat: 50.08328, lon: 19.95236, speed: 10.0, course: 229.0
2024-04-19 11:13:48  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031113471904245004987200019571309e024214ffffffffff041c000000000001cc00ff0000e900980000000029
2024-04-19 11:13:48  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:13:47, lat: 50.08312, lon: 19.95218, speed: 24.0, course: 214.0
2024-04-19 11:13:55  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031113541904245004917300019570868e037202ffffffffff041b000000000001cc00ff0000e80099000000002a
2024-04-19 11:13:55  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:13:54, lat: 50.08196, lon: 19.95145, speed: 37.0, course: 202.0
2024-04-19 11:14:10  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114091904245004764500019569830e039204ffffffffff041f000000000001cc00ff0000e7009a000000002b
2024-04-19 11:14:25  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114241904245004636100019568896e030207ffffffffff041f000000000001cc00ff0000e50099000000002c
2024-04-19 11:14:37  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114361904245004551800019568133e027235ffffffffff0419000000000001cc00ff0000df0098000000002d
2024-04-19 11:14:38  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114371904245004550600019568023e024254ffffffffff0419000000000001cc00ff0000de0099000000002e
2024-04-19 11:14:39  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114381904245004552400019567896e025269ffffffffff0419000000000001cc00ff0000e0009f000000002f
2024-04-19 11:14:40  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114391904245004553100019567785e023270ffffffffff0418000000000001cc00ff0000e200990000000030
2024-04-19 11:14:46  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114451904245004554800019567359e011237ffffffffff041b000000000001cc00ff0000e200980000000031
2024-04-19 11:14:46  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114461904245004552900019567323e012207ffffffffff041b000000000001cc00ff0000e200980000000032
2024-04-19 11:14:48  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114481904245004546000019567303e019170ffffffffff041a000000000001cc00ff0000e100980000000033
2024-04-19 11:14:54  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114541904245004473700019567481e054165ffffffffff041c000000000001cc00ff0000de00980000000034
2024-04-19 11:14:58  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031114571904245004432100019567719e052152ffffffffff041c000000000001cc00ff0000dc00980000000035
2024-04-19 11:15:11  INFO: [T08600000: h02 < 37.???.???.???] 2a48512c373334303030333730332c56312c3131313530392c412c353030342e333036342c4e2c30313935362e393031392c452c3030302e30302c3030302c32
2024-04-19 11:15:11  INFO: [T08600000: h02 < 37.???.???.???] 31303432342c42464646464646462c3436302c30302c302c302c3623
2024-04-19 11:15:11  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031115091904245004306400019569019e039146ffffffffff0416000000000001cc00ff0000e000990000000036
2024-04-19 11:15:11  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:15:09, lat: 50.07177, lon: 19.94837, speed: 39.0, course: 146.0
2024-04-19 11:15:18  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031115171904245004236700019569620e036168ffffffffff041f000000000001cc00ff0000e300980000000037
2024-04-19 11:15:18  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:15:17, lat: 50.07061, lon: 19.94937, speed: 36.0, course: 168.0
2024-04-19 11:15:25  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031115241904245004167200019569720e033167ffffffffff041f000000000001cc00ff0000e000980000000038
2024-04-19 11:15:25  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:15:24, lat: 50.06945, lon: 19.94953, speed: 33.0, course: 167.0
2024-04-19 11:15:28  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031115271904245004141900019569855e030151ffffffffff041f000000000001cc00ff0000dd00980000000039
2024-04-19 11:15:28  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:15:27, lat: 50.06903, lon: 19.94976, speed: 30.0, course: 151.0
2024-04-19 11:15:31  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031115301904245004121300019570102e031131ffffffffff041f000000000001cc00ff0000dc0098000000003a
2024-04-19 11:15:31  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:15:30, lat: 50.06869, lon: 19.95017, speed: 31.0, course: 131.0
2024-04-19 11:15:40  INFO: [T08600000: h02 < 37.???.???.???] 2470000017031115391904245004096600019570749e006119ffffffffff0419000000000001cc00ff0000e10098000000003b
2024-04-19 11:15:40  INFO: [T08600000] id: 7000001703, time: 2024-04-19 11:15:39, lat: 50.06828, lon: 19.95125, speed: 6.0, course: 119.0

the part between 11:14:10 and 11:15:11 seems to be ignored for some reason. The device is Micodus MV730G.

tananaev commented 5 months ago

Are you sure you haven't filtered the logs?

I have tested some of the samples and they're decoded just fine.

kusmierz commented 5 months ago

Hopefully not. That certainly wasn't my intention. My config file is:

# rg filter conf/traccar.xml
35:  <entry key="filter.enable">true</entry>
36:  <entry key="filter.invalid">true</entry>
37:  <entry key="filter.zero">true</entry>
tananaev commented 5 months ago

Have you tried without filtering?

kusmierz commented 5 months ago

yes, initially I was using Traccar without filter, but after this (zero point) I have added some as mentioned above.

image

what's more, it seems to be logged properly:

tracker-server.log.20240420:2024-04-20 07:00:59  INFO: Position filtered by Invalid filters from device: 017anotherdevice
tracker-server.log.20240420:2024-04-20 07:21:18  INFO: Position filtered by Invalid filters from device: 017anotherdevice
tracker-server.log.20240420:2024-04-20 14:26:32  INFO: Position filtered by Invalid Zero Future filters from device: 7000001703
tananaev commented 5 months ago

Let's stick to the topic. Is everything decoded when you disable filtering?

kusmierz commented 5 months ago

Sorry for not being clear earlier. I meant to explain that I hadn't noticed this behaviour when not using the filter. I didn't use that configuration for very long though, because it started to show some zero points. That's why I switched on the filter. I can turn it off for a longer period and check if the problem still occurs.

tananaev commented 5 months ago

Let's do that. For now I'm going to close the ticket.

kusmierz commented 5 months ago

That was quick. After one trip without filtering (all filters turned off in the config), I've got this weird missing entries despite proper entries in the logs (same as above). Additionally I've got zero point, which messed my totalDistance et all (I'm mentioning this only to show that filters were disabled).

tananaev commented 5 months ago

Send us the full logs with filtering disabled.

kusmierz commented 5 months ago

sent to support@traccar.org (with almost no obfuscation).

look at 15:42:55 - 15:44:57

tananaev commented 5 months ago

Looks like you just need to set h02.messageLength.

kusmierz commented 5 months ago

Thank you @tananaev! Could you help me determine the value? I don't quite understand what length to choose...

[protocol].messageLength config

Custom message length. Currently used only by H2 protocol for specifying binary message length.
tananaev commented 5 months ago

The value should match the length of the binary messages.

kusmierz commented 5 months ago

ok, so based on this and that (and my understanding of the protocol and this configuration), I assumed that it needs to be set to 51 not 45 as in example, right?

❯ rg h02 logs/tracker-server.log* | rg ' 24[a-f0-9]+' | cut -d ' ' -f 9 | awk '{print length}' | sort | uniq -c
   2979 102
      1 204
  <entry key='h02.messageLength'>51</entry>
tananaev commented 5 months ago

I see 51.