Holux RCV-3000 - all export does not work - GPX export for OSM don't work (with debug logs)

[yannick56]

Hello,

Exporting data to GPX format worked fine in September 2016 for my first generation Holux RCV-3000 (using a Mediatek MT-3329 chip for GPS only) with: Mageia Linux 5.0 + gpsd v3.11 + GPSBabel 1.5.0 and the following command line: gpsbabel -t -i m241 -f /dev/ttyUSB0 -o gpx -F $(date -Iminutes).gpx

But I did some tests the last weekend with the following 2 more recent configurations and it doesn't work at all anymore:

• Mageia Linux 9.0 + gpsd v3.25 + GPSBabel 1.8.0.
• Windows 10 (with update 22H2) + LOCOSYS Technology GPS Receiver Drivers version 1.10.03.22 + GPSBabel 1.9.0. and I have a lot of error messages when exporting to GPX format or all other formats. When it retrieves each position of a path, there is an error like this: gpsbabel -t -i m241 -f COM7 -o gpx -F C:/Users/utilisateur/Documents/gpsbabel/holux-holuxm241-export.gpx 0: Bad CRC 40 != 0a (pos 0x000470) 1: Bad CRC 00 != c4 (pos 0x000470) 2: Bad CRC 5b != 8e (pos 0x000470) 3: Bad CRC 42 != 19 (pos 0x000470) 4: Bad CRC 00 != 80 (pos 0x000470)

Attached are debug log files of the export to GPX format for the 2 configurations indicated above. Can you fix the problem with this information?

PS: The second generation Holux RCV-3000 uses a Mediatek MT-3333 chip (GPS + GLONASS) and requires the use of MS Windows drivers for Silicon Labs CP210x.

Thanks in advance.

[yannick56]

gpsbabel19-win10-holuxm241-gpx.txt gpsbabel19-win10-holuxm241-osm.txt gpsbabel19-win10-mtklogger-gpx.txt gpsbabel19-win10-mtklogger-osm.txt holux-holuxm241-export.gpx.txt holux-holuxm241-osm-data.osm.txt holux-mtklogger-export.gpx.txt holux-mtklogger-osm-data.osm.txt

[yannick56]

debug log file ans output exported files

[robertlipe]

Does 1.5.0 read the current device or is the flash image corrupt? That a common fault in this class of devices.

On Tue, Aug 27, 2024, 5:26 PM Yannick @.***> wrote:

debug log file ans output exported files

— Reply to this email directly, view it on GitHub https://github.com/GPSBabel/gpsbabel/issues/1330#issuecomment-2313681508, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCSD377GDUNDQBRWP5GFCLZTT4KHAVCNFSM6AAAAABNHBAQOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJTGY4DCNJQHA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[yannick56]

1)

Does 1.5.0 read the current device

I have no old linux distribution with GPSBabel 1.5.x for testing. I found no GPSBabel 1.5.x windows binaries on Internet to download for testing with Holux RCV-3000 (with MT-3329 chip). Can you provide me a link/URL to download GPSBabel 1.5.x for Ms Windows for testing goal ?

2)

or is the flash image corrupt?

The flash image in my Holux RCV-3000 gps logger device is NOT corrupt because I can download the gps data in the memory in the device by using BT747 v2.3.2 (file BT747_2.3.2_en_full.zip) or BT747 v2.1.7 (file BT747_2.1.7_full.zip) with Mageia Linux 9.0 + gpsd v3.25 + rxtx lib. The BT747 zip files are available at https://sourceforge.net/projects/bt747/files/Development/. The BT747 software does NOT work with MS Windows because it needs the very old Java SE 5 (alias 1.5) (2004-2009) to work on MS Windows. When you launch the BT747 gui (.exe) ou command line tool (.exe), there is a popup window box with the text "This application requires a Java Runtime Environment 1.5.0". (https://en.wikipedia.org/wiki/Java_version_history#Java_5 ).

For many OSM (OpenStreetMap) contributors, GPSBabel is maybe the de facto standard for uploading GPS traces : https://wiki.openstreetmap.org/wiki/List_of_GPS_trace_file_formats

[tsteven4]

You can build something like 1.5.0 on Ubuntu jammy with the attached patch and apt-get install qtbase5-dev git checkout 1af59ed cd gpsbabel CFLAGS=-fPIC ./configure make

gpsbabel_150_jammy.patch

[tsteven4]

I did verify that all the nmea checksums in the "Read ..." messages in gpsbabel19-win10-holuxm241-osm.txt are correct. It isn't clear to me what is going wrong but I think we are lost when we report "Bad CRC".

It might be helpful to have the binary file created by the attempt to read the device (from the log this was C:/Users/utilisateur/AppData/Local/Temp\data.bin) as well as the log from when that file was created.

[tsteven4]

I did find that gpsbabel 1.4.3 is available on Ubuntu trusty tahr. https://releases.ubuntu.com/trusty/

[tsteven4]

We seem to be lost by

Opening file C:/Users/utilisateur/AppData/Local/Temp\data.bin...
Default Bitmask 01068800, Log every 0 sec, 0 m, 429490176 km/h
Using initial bitmask 01068800 for parsing the .bin file

The bitmask isn't believable, and the log intervals seem unlikely.

[yannick56]

1)

I build gpsbabel 1.5.0 (source code from https://github.com/GPSBabel/gpsbabel/releases/tag/gpsbabel_1_5_0 ) with your patch [gpsbabel_150_jammy.patch] and cmake 3.26.4 and lib64qt5core-devel 5.15.7 and lib64qt5gui-devel 5.15.7. And i have the same problem:

Data block:57 54 35 5f 74 17 40 42 db

0: Bad CRC 40 != 0a (pos 0x000470) Entering mtk_parse, count = 1, dataLen = 9

Data block:42 db cf 5c c0 0e 00 00 00

1: Bad CRC 00 != c4 (pos 0x000470) Entering mtk_parse, count = 2, dataLen = 9

see log gpsbabel150-linux-mageia9-holuxm241-gpx.txt and data.bin file

2) September 10, 2016, I wrote here https://wiki.openstreetmap.org/wiki/Holux_RCV-3000#GPSBabel after testing that all worked fine:

_With command line cd /tmp gpsbabel -t -i m241 -f /dev/ttyUSB0 -o gpx -F $(date -Iminutes).gpx ls and you should have a file like this 2016-09-10TO4:56+0200.gpx

Notes: To get data from RCV-3000, use only the parameter -i m241, all the others for holux, mediatek, nmea, etc ...does not work._

Then now, I can not understand why gpsbabel 1.5.0 does not work with Linux or MS Windows. For Linux, is there a problem with gpsd changes from 3.11 to 3.25.

[yannick56]

gpsbabel150-linux-mageia9-holuxm241-gpx.txt

[yannick56]

2024-08-30T02:18+02:00.gpx.txt

[yannick56]

Linux - gpsbabel 1.5.0 - data.bin data.bin.txt

[tsteven4]

gpsbabel and mtkbabel do similar things with your binary file. both think we are logging VDOP,AZIMUTH,RCR,MILLISECOND which doesn't make much sense. Perhaps you can try to create a bin file with mtkbabel and see if that works.

./mtkbabel  -d 7 -b ../base/yannick/data.bin -c
Reading offset 00000000

Sector header:      3D00008806011E0000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2ABFBBBBBBBB
Record count:        3D00        61 records
Log format mask:     00880601    00000001000001101000100000000000b (VDOP,AZIMUTH,RCR,MILLISECOND)
Log mode mask:       1E00        0000000000011110b (AUTOLOG_ON,STOP_WHEN_FULL)
Log period:          00000000      0.00 s
Log distance:        00000000      0.00 m
Log speed:           0000FFFF    429490176.00 km/h
Failed sectors mask: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00FF

Separator: AAAAAAAAAAAAAA0704010000BBBBBBBB, type: CHANGE_START_STOP_LOG
Reading offset 00000210
Separator: AAAAAAAAAAAAAA0332000000BBBBBBBB, type: CHANGE_LOG_PERIOD
Reading offset 00000220
Separator: AAAAAAAAAAAAAA04E8030000BBBBBBBB, type: CHANGE_LOG_DISTANCE
Reading offset 00000230
Separator: AAAAAAAAAAAAAA031E000000BBBBBBBB, type: CHANGE_LOG_PERIOD
Reading offset 00000240
Separator: AAAAAAAAAAAAAA040A000000BBBBBBBB, type: CHANGE_LOG_DISTANCE
Reading offset 00000250
Separator: AAAAAAAAAAAAAA031E000000BBBBBBBB, type: CHANGE_LOG_PERIOD
Reading offset 00000260
Separator: AAAAAAAAAAAAAA0400000000BBBBBBBB, type: CHANGE_LOG_DISTANCE
Reading offset 00000270
Separator: HOLUXGR241LOGGER
Setting log format and data types for Holux, model ID 0043
Reading offset 00000284
Separator: AAAAAAAAAAAAAA0706010000BBBBBBBB, type: CHANGE_START_STOP_LOG
Reading offset 00000294
Reading log sector: record 1 (1/4294967295 total)
Record VDOP: 5754 (215.91)
Record RCR: 355F (TIME,DISTANCE)
Record MILLISECOND: 7417 (6004)
ERROR: Record checksum error: expected 0x40, computed 0x0A

./bld/gpsbabel -D 9 -i m241-bin -f yannick/data.bin 2>&1 | more
GPSBabel Version: 1.9.0
main: Compiled with Qt 6.2.4 for architecture x86_64-little_endian-lp64
main: Running with Qt 6.2.4 on Ubuntu 22.04.4 LTS, x86_64
main: QLocale::system() is C
main: QLocale() is C
main: QTextCodec::codecForLocale() is UTF-8, mib 106
Opening file yannick/data.bin...
Default Bitmask 01068800, Log every 0 sec, 0 m, 429490176 km/h
Using initial bitmask 01068800 for parsing the .bin file
# GPS Logger# Log disabled
# Log period change 5 sec
# Log distance change 100.0 m
# Log period change 3 sec
# Log distance change 1.0 m
# Log period change 3 sec
# Log distance change 0.0 m
log len VDOP 11 1068800
log len AZIMUTH 15 1068800
log len RCR 17 1068800
log len MILLISECOND 18 1068800
log len  24 1068800
mtk_logger: Unknown size/meaning of bit 24
Log item size 9 bytes
Log item size 9 bytes
# GPS Logger# Turned On
Entering mtk_parse, count = 0, dataLen = 9
# Data block:57 54 35 5f 74 17 40 42 db
40 a
 0: Bad CRC 40 != 0a (pos 0x000470)

[GPSBabelDeveloper]

OK, so GPSBabel 1.5.0 was never able to read that device in its current state. This isn't a case where we broke something; this never worked for this pile of bits from any device.

Looking at the dump, it's not flagrantly corrupted (header block of 0xaa's starts at 0x200, etc.) so it's not like the device driver totally mangled the serial stream beyond recognition or something, It's just .... wrong.

Perhaps a firmware update or something in the intervening years changed the way the records are stored in flash, This happened at least once in MTK history.

We're able to read the very very early part of this as an MTK image, but it very quickly just falls apart.

unsigned int log_period = le_read32(buf + 8);
unsigned int log_distance = le_read32(buf + 12);
unsigned int log_speed = le_read32(buf + 16);

Those are all zeros in this image. When it tries to read the bitmasks and the sizes to get the record strides, it's just nonsense. That crazy speed is 2^32, which is, of course, nonsensical. The wheels fall off the bus long before we get into individual points.

Perhaps someone else is willing/able to basically reverse engineer what I suspect is a corrupt file and rescue some amount of data from it - and there probably is something as there's a large chunk of tell-tale "barber pole" pattern when viewed as a hex dump - but I'm not going to be able to essentially reverse compile it from the numerology.

This data is corrupt. GPSBabel 1.5.0 didn't read this. If you have some other program that can read it, great. Perhaps you can find what it knows that we don't and graft that knowledge into mtk_logger.cc, but I've spent about all the time I'm able to on it. Holux is out of business, so the population of these loggers is only shrinking. We don't know why your device is unique.

Sorry for the bad news. RJL

On Thu, Aug 29, 2024 at 7:34 PM Yannick @.***> wrote:

Linux - gpsbabel 1.5.0 - data.bin data.bin.txt https://github.com/user-attachments/files/16808712/data.bin.txt

— Reply to this email directly, view it on GitHub https://github.com/GPSBabel/gpsbabel/issues/1330#issuecomment-2319552646, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC3VAD4WKTIBR5EBG36JYZDZT64Z5AVCNFSM6AAAAABNHBAQOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJZGU2TENRUGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[tsteven4]

I was able to recover what I think is almost all the data in

Linux - gpsbabel 1.5.0 - data.bin data.bin.txt

As I suspected the bitmask was incorrect. I inserted two 0xff bytes at the beginning of the file, and then gpsbabel -t -D 5 -i m241-bin -f data_insert.bin -o kml -F data_insert.kml 2>data_insert.log seems to recover almost all the data.

The modified bin file, the debug log, and data converted to kml are included: yannick.zip

The last 4 log entries report Bad CRCs (1949-1952).

It would have been interesting to have seen the attempt to read the device with -D 9 when this bin file was created. That should have shown us if the two missing bytes were in the NMEA sentence sent by the device. However, in another debug log gpsbabel19-win10-holuxm241-osm.txt we can see they are missing. Furthermore, that sentence had a valid checksum so it is unlikely to have been corrupted in transmission. It appears your device is not including the first two bytes when it transmits the log via NMEA.

It also would be interesting if you can recover the log data in a binary file via some other means, e.g. BT747.

[tsteven4]

Furthermore, that sentence had a valid checksum so it is unlikely to have been corrupted in transmission.

Actually the NMEA checksum is unchanged by the deletion of the first two bytes if their value was 0xff, so it is possible they were dropped in transmission.

[tsteven4]

Although it is more involved, I can recover the same data from gpsbabel19-win10-holuxm241-osm.txt that I recovered from Linux - gpsbabel 1.5.0 - data.bin data.bin.txt

Both gpsbabel 1.5.0 and 1.9.0 think that the first 16 bytes of your log memory starting at address 0 are somehow shifted left two bytes.

[yannick56]

Bt747 v2.3.2 screenshot for Device Settings tab. There is buuton for "Erase only" device memory or "Try to rrecover faulty memory" or "Set Format and Erase".

[yannick56]

gpsbabel150-linux-mageia9-holuxm241-gpx-after-cleaning-memory-20240901.txt

[yannick56]

In Bt747 v2.3.2 screenshot for Device Settings tab. There is buuton for "Erase only" device memory or "Try to rrecover faulty memory" or "Set Format and Erase".

I click on all these 3 buttons to clean records in memory and start a new record track. Then now, it works better with gpsbabel 1.5.0 withe several errors (like this 168: Bad CRC 70 != 1b (pos 0x001400) ) , see the log gpsbabel150-linux-mageia9-holuxm241-gpx-after-cleaning-memory-20240901.txt

[yannick56]

data.bin.gpsbabel1.5.0-20240901.txt

[yannick56]

It also would be interesting if you can recover the log data in a binary file via some other means, e.g. BT747.

Here what you ask.

BT747-v2.3.2-log-20240901.bin.txt

[yannick56]

[yannick56]

[tsteven4]

It also would be interesting if you can recover the log data in a binary file via some other means, e.g. BT747.

Here what you ask.

Uploading BT747-v2.3.2-log-20240901.bin.txt…

This is a link to this issue, not a file.

[tsteven4]

I can see there were serial communication issues recorded in gpsbabel150-linux-mageia9-holuxm241-gpx-after-cleaning-memory-20240901.txt. May I suggest you replace the battery in your device with a fresh one?

[tsteven4]

These serial communication issues were not detected by gpsbabel and lead to the "Bad CRC" messages and some corrupt points in the output file.

[yannick56]

It also would be interesting if you can recover the log data in a binary file via some other means, e.g. BT747.

Here what you ask. Uploading BT747-v2.3.2-log-20240901.bin.txt…

This is a link to this issue, not a file.

I have uploaded the 4 Mo file.

[yannick56]

I can see there were serial communication issues recorded in gpsbabel150-linux-mageia9-holuxm241-gpx-after-cleaning-memory-20240901.txt. May I suggest you replace the battery in your device with a fresh one?

The RCV-3000 is connected to the computer with USB cable (with power button is ON position) and without battery the device can work but with power from USB cable.

[tsteven4]

The BT747 binary memory image looked good, and the current gpsbabel can convert the memory image correctly.

gpsbabel 1.5.0 had trouble reading the device memory that BT747 did not. The serial read of memory in the current gpsbabel is a bit different, but I don't know that it would fair any better. To improve the situation would require a volunteer with a device to investiage further. It is clear that the serial read for mtk loggers could be better (e.g verify nmea checksums, verify expected length of data returned in PMTK182,8 sentences, verify no holes in recovered memory image). As Robert noted, the manufacturer has been out of business for 5 years and interest in this format is declining.

At this point I don't see any evidence that the current version of gpsbabel is any worse than 1.5.0.