Closed W-Geronius closed 1 month ago
'no GPS data' arises as soon as date->valid == false. See OBP60Extensions.cpp lines ~210 ff. To understand further logs, I introduced debug statements there: LOG_DEBUG(GwLog::ERROR,"... OBPExtensions: GPS lost") LOG_DEBUG(GwLog::ERROR,"... OBPExtensions: GPS retrieved")
date is invalidated by main esp-nmea2000 code. As to Andreas, only timestamps are compared and set invalid if unchanged for >= 4 seconds. Invalid NMEA records do not result in invalid date1 To understand further logs, I introduced debug statements in main.cpp after nmea0183Converter->loop(), line ~833: logger.logDebug(GwLog::ERROR,"main.cpp: GPST invalid") logger.logDebug(GwLog::ERROR,"main.cpp: GPST recovered")
Initially, GPS data (nmea0183) were received from OP/SignalK. Network problems were investigated and appeared to be the reason:
No invalid dates occurred on M5Stack-Atom, fed by SK on Raspi via WiFi
'no GPS data' occurs on plain OBP60 as well: no external input/output Page 1 set to GPST. No other pages defined and populated. GPS data originate from internal GPS !!
according to log page is drawn every second, on screen however seconds are incremented in 2 second steps!
[2022-05-31 09:17:06] GPST freezes [2022-05-31 09:17:10] GWSERIAL:79149:main.cpp: GPST invalid [2022-05-31 09:17:11] GWSERIAL:80064:... OBPExtensions: GPS lost ... [2022-05-31 09:17:15] GWSERIAL:83883:main.cpp: GPST recovered [2022-05-31 09:17:16] GWSERIAL:84642:... OBPExtensions: GPS retrieved
same happens right again at: [2022-05-31 09:17:19]
detailed log below, NMEA0183 message entries: api->sendNMEA0183Message(NMEA0183Msg); line 279 in OBPSensorTask.cpp
screenshot-video and logs covering a 'GPS no data' event at 14:08:09 UTC
Not completely sure what you have logged in the raw...log. But it looks rather strange. Just only checked the RMC records. Typically there are 2 of them - with different format.
[2022-05-31 16:08:05] $GPRMC,140803.00,A,5043.910,N,700.479,E,0.4,0.0,310522,0.0,E*52
[2022-05-31 16:08:05] $GPRMC,140804.00,A,5043.91031,N,00700.47982,E,0.470,,310522,,,A*72
Where do they come from? And at the first GPS lost
[2022-05-31 16:08:15] GWSERIAL:470587:... OBPExtensions: GPS lost
The records are really strange:
[2022-05-31 16:08:11] $GPRMC,140810.00,A,5043.91344,N,00700.48139,E,0.380,,310522,,,A*79
[2022-05-31 16:08:11] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:11] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:12] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:13] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:13] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:14] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:15] $GPRMC,140810.00,A,5043.913,N,700.481,E,0.4,0.0,310522,0.0,E*54
[2022-05-31 16:08:17] $GPRMC,140816.00,A,508,1.44*0C
[2022-05-31 16:08:17] $GPRMC,140815.00,A,5043.914,N,700.482,E,,,,,*3D
[2022-05-31 16:08:17] $GPRMC,140815.00,A,5043.914,N,700.482,E,,,,,*3D
So we see that the GPS time does not evolve, then we get an invalid record (140816) with a timestamp more in the future then the next ones. Most probably the RMC that we still see is the one that is automatically send by the core every 500 ms. But this mean that the GPS really does either not send data - or that some data is missing (buffer overflow or similar). Something really strange is the invalid RMC with 140816.00 - this is nothing we somehow generate - so could really be a problem in the GPS.
BTW: Does the same happen with disabled internal GPS?
thanks Andreas for jumping in!
Not completely sure what you have logged in the raw...log.
It's the MFD's TCP server 0183 output as seen on default port 10110. This seemed to be the only possibility to examine the MFD's (internal) NMEA 0183 records without disrupting the system by excessive logging.
Does the same happen with disabled internal GPS?
of course there is no 0183 output at all when internal GPS is disabled. Feeding external GPS data introduces alien hardware, software and possibly whacky data transport. That's the route I took when started to analyze this bug - the current setup can be easily reproduced by anyone
Observing the time pattern of 'no GPS data' events, I have a strong gut feeling that we are seeing the very same issue when using any external 0183 GPS sources - this favours your hypothesis of buffer overflow or alike.
RMC records. Typically there are 2 of them - with different format. Where do they come from? ... So we see that the GPS time does not evolve, then we get an invalid record (140816) with a timestamp more in the future then the next ones.
Excellent - I haven't spotted that so far. Maybe Norbert can shed some light on that as I am quite limited in understanding the tasks involved.
RMC records. Typically there are 2 of them - with different format. Where do they come from? ... So we see that the GPS time does not evolve, then we get an invalid record (140816) with a timestamp more in the future then the next ones.
Just to verify I ran another test - MFD fed by Raspberry/SK/Moitessier GPS data (logs available). There is just a single format of RMC records with no records out of time order. This might indeed be an issue of the MFD internal GPS - 'no GPS data' continue to appear in the very same manner, so it's an issue not related to this
Do we have logs for this second scenario?
Here they are: com6 is the display's logging output, telnet is the display's complete 0183 output. I added a 'combined' file, merging log output and RMC output sorted by log time. 10.10.10.81 is the PC listening and recording the display's output:
We must have bought fake China chips again. The UBlox NEO-6M is probably an ATGM332D that should be pin-compatible with the UBlox NEO-6N. The AT6558 chip is installed in the ATGM332D module. This can be recognized by the first lines of output immediately after switching on. It is quite possible that the ATGM332D shows such erratic behavior.
More details can be found here: https://www.youtube.com/watch?v=buyFIcgee5w
no GPS dropouts were observed during 7 days live on board using a different OBP60 with no GPS and BME280 installed. This supports Norbert's hypothesis. Occasional reboots however still occurred.
In my current setup I have 2 OBP60 installed, data fed from same SK on RPi4. The device with GPS/BMP280 installed (but deactivated in config) again displays dropouts, the device without those sensors runs smoothly
The problem is a software timing in the GPS data reading routine. Only all 1s read the GPS data. The data rate form GPS has the same rate. The data transmission is asynchronous and produce the problems. With al reading rate from 2 Hz is all good. The problem is fixed in the new firmware dev20240920 for the new hardware V2.1
This bug has been hunted for since end of march - the reason unfortunately remains unclear. This issue is created to summarize different findings in a single place.
Sporadically, 'no GPS data' is displayed, time and position data disappear. Signal usually is recovered within a few seconds. No obvious pattern