semuconsulting / pyubx2

Python library for parsing and generating UBX GPS/GNSS protocol messages.
BSD 3-Clause "New" or "Revised" License
173 stars 67 forks source link

UBX-ESF-STATUS messages seem incorrect #46

Closed pezosanta closed 2 years ago

pezosanta commented 2 years ago

I have a NEO-M8U-0-10 module and want to monitor the status of the sensor fusion calibration process with a python script running on a Windows 10 desktop (with USB serial connection).

However, the parsed ESF-STATUS messages (shown below) differ from the UBX-ESF-STATUS messages shown in the documentation.

<UBX(ESF-STATUS, iTOW=17:20:34.350000, version=2, reserved1=36, reserved2=2, reserved3=0, status=0, reserved4=0, reserved5=0, numCh=6, type_01=5, used_01=0, ready_01=1, calibStatus_01=0, timeStatus_01=1, freq_01=0, badMeas_01=1, badTTag_01=0, missingMeas_01=1, noisyMeas_01=1, type_02=4, used_02=0, ready_02=0, calibStatus_02=2, timeStatus_02=2, freq_02=142, badMeas_02=0, badTTag_02=0, missingMeas_02=1, noisyMeas_02=0, type_03=10, used_03=0, ready_03=0, calibStatus_03=0, timeStatus_03=0, freq_03=4, badMeas_03=0, badTTag_03=1, missingMeas_03=0, noisyMeas_03=1, type_04=0, used_04=0, ready_04=0, calibStatus_04=1, timeStatus_04=0, freq_04=10, badMeas_04=0, badTTag_04=0, missingMeas_04=0, noisyMeas_04=0, type_05=18, used_05=0, ready_05=1, calibStatus_05=0, timeStatus_05=1, freq_05=0, badMeas_05=0, badTTag_05=0, missingMeas_05=0, noisyMeas_05=0, type_06=0, used_06=0, ready_06=0, calibStatus_06=0, timeStatus_06=0, freq_06=0, badMeas_06=0, badTTag_06=0, missingMeas_06=0, noisyMeas_06=0)>

In the message payload:

To reproduce this behaviour use the following code snippet:

from serial import Serial
from pyubx2 import UBXReader

stream = Serial('COM5', 115200, timeout=3)
ubr = UBXReader(stream)

while True:
    (raw_data, parsed_data) = ubr.read()

    if parsed_data.identity == 'ESF-STATUS':
        print(parsed_data)
semuadmin commented 2 years ago

Hi @pezosanta ,

Thanks for bringing this to my attention, and sorry for the trouble you've been having.

I've just reviewed the documentation for ESF-STATUS from various generations of u-blox device. Some of these issues appear to be cosmetic changes from older documentation e.g. the status field is equivalent to fusionMode, numCh is equivalent to numSens, and the various reserved fields have different names in older documentation but occupy the same payload positions. There does however appear to be a genuine error in my definition of the sensStatus2 bitfield, which I have as X2 but which should be X1. I'm guessing this accounts for the payload discrepancies you highlight above, at least for the 2nd and subsequent sensor readings.

I believe the correct definition should be as follows:

    "ESF-STATUS": {
        "iTOW": U4,
        "version": U1,
        "reserved0": U7,
        "fusionMode": U1,
        "reserved1": U2,
        "numSens": U1,
        "group": (
            "numSens",
            {  # repeating group * numSens
                "sensStatus1": (
                    X1,
                    {
                        "type": U6,
                        "used": U1,
                        "ready": U1,
                    },
                ),
                "sensStatus2": (
                    X1,
                    {
                        "calibStatus": U2,
                        "timeStatus": U2,
                    },
                ),
                "freq": U1,
                "faults": (
                    X1,
                    {
                        "badMeas": U1,
                        "badTTag": U1,
                        "missingMeas": U1,
                        "noisyMeas": U1,
                    },
                ),
            },
        ),
    },

I've applied this change and tested against your original message payload and the results appear to be consistent with your expectations (the iTOW will be different to yours):

<UBX(ESF-STATUS, iTOW=23:59:42, version=2, reserved0=548, fusionMode=0, reserved1=0, numSens=6, type_01=5, used_01=0, ready_01=1, calibStatus_01=0, timeStatus_01=1, freq_01=0, badMeas_01=0, badTTag_01=0, missingMeas_01=0, noisyMeas_01=0, type_02=13, used_02=0, ready_02=0, calibStatus_02=0, timeStatus_02=1, freq_02=10, badMeas_02=0, badTTag_02=0, missingMeas_02=0, noisyMeas_02=0, type_03=14, used_03=0, ready_03=1, calibStatus_03=0, timeStatus_03=1, freq_03=10, badMeas_03=0, badTTag_03=0, missingMeas_03=0, noisyMeas_03=0, type_04=0, used_04=0, ready_04=0, calibStatus_04=0, timeStatus_04=1, freq_04=10, badMeas_04=0, badTTag_04=0, missingMeas_04=0, noisyMeas_04=0, type_05=1, used_05=0, ready_05=0, calibStatus_05=0, timeStatus_05=0, freq_05=10, badMeas_05=0, badTTag_05=0, missingMeas_05=0, noisyMeas_05=0, type_06=18, used_06=0, ready_06=1, calibStatus_06=0, timeStatus_06=1, freq_06=0, badMeas_06=0, badTTag_06=0, missingMeas_06=0, noisyMeas_06=0)>

The type fields for the six sensor readings come out as (in sequence) 5, 13, 14, 0, 1 and 18.

I believe the complete list of supported sensor types is as follows (not all apply to the M8U, and note that some of the values have scaling factors applied):

Type Description Unit Format of the 24 data bits
0 none, data field contains no data
1...4 reserved
5 z-axis gyroscope angular rate deg/s *2^-12 signed
6...7 reserved
8 rear-left wheel ticks Bits 0-22: unsigned tick value. Bit 23:
direction indicator (0=forward, 1=backward)
9 rear-right wheel ticks Bits 0-22: unsigned tick value. Bit 23:
direction indicator (0=forward, 1=backward)
10 single tick (speed tick) Bits 0-22: unsigned tick value. Bit 23:
direction indicator (0=forward, 1=backward)
11 speed m/s * 1e-3 signed
12 gyroscope temperature deg Celsius * 1e-2 signed
13 y-axis gyroscope angular rate deg/s *2^-12 signed
14 x-axis gyroscope angular rate deg/s *2^-12 signed
15 reserved
16 x-axis accelerometer-specific force m/s^2 *2^-10 signed
17 y-axis accelerometer-specific force m/s^2 *2^-10 signed
18 z-axis accelerometer-specific force m/s^2 *2^-10 signed
19...63 reserved

If you have any further problems, please don't hesitate to get back to me.

pezosanta commented 2 years ago

Hi @semuadmin,

Sorry for the late reply, I was focusing on my other job tasks and had some days off as well.

Thank You for your amazingly rapid bug-fix ! 🥇 Now the UBX-ESF-STATUS messages seem to be correct.

<UBX(ESF-STATUS, iTOW=10:20:42.950000, version=2, reserved0=548, fusionMode=0, reserved1=0, numSens=6, type_01=5, used_01=0, ready_01=1, calibStatus_01=0, timeStatus_01=1, freq_01=10, badMeas_01=0, badTTag_01=0, missingMeas_01=0, noisyMeas_01=0, type_02=13, used_02=0, ready_02=1, calibStatus_02=0, timeStatus_02=1, freq_02=10, badMeas_02=0, badTTag_02=0, missingMeas_02=0, noisyMeas_02=0, type_03=14, used_03=0, ready_03=1, calibStatus_03=0, timeStatus_03=1, freq_03=10, badMeas_03=0, badTTag_03=0, missingMeas_03=0, noisyMeas_03=0, type_04=16, used_04=0, ready_04=1, calibStatus_04=0, timeStatus_04=1, freq_04=10, badMeas_04=0, badTTag_04=0, missingMeas_04=0, noisyMeas_04=0, type_05=17, used_05=0, ready_05=1, calibStatus_05=0, timeStatus_05=1, freq_05=10, badMeas_05=0, badTTag_05=0, missingMeas_05=0, noisyMeas_05=0, type_06=18, used_06=0, ready_06=1, calibStatus_06=0, timeStatus_06=1, freq_06=10, badMeas_06=0, badTTag_06=0, missingMeas_06=0, noisyMeas_06=0)>

However, I have some questions (I'm new to the Ublox universe) about

  1. How to check current UBX-CFG configs and how to modify them if needed
  2. Whether you have any example codes of applying "Save On Shutdown (SOS) Feature" (which requires a sequence of configuration actions). This would be useful to save the calibrated ESF (IMU) parameters between receiver restarts. However, the documentation says that this can be achieved with SOS or if non-volatile storage is available. So I'm little bit confused whether using SOS is necessary in my case (NEO-M8U-0-10 module) or not.

If you have some spare time to help me out, should I create a new "Issue" with my questions, or reach you out on another platform (e.g. email) instead?

semuadmin commented 2 years ago

Hi @pezosanta

Your latest questions should probably be raised on the u-blox support forum https://portal.u-blox.com/s/. Please appreciate that I don't officially represent u-blox and the work on pyubx2 and its companion libraries pygpsclient and pynmeagps are done on an entirely unpaid voluntary basis (though you are welcome to make a small donation if you find the library useful and wish to encourage its future development and support!).

Having said that, if it's a Python 3 implementation you're after, the official u-blox support forum will probably point you back in my direction :-)

pyubx2 and the companion PyGPSClient GUI fully support "UBX-CFG" configuration on all u-blox GNSS devices from Generation 6 onwards and you'll find some implementation examples in the \examples folder. Note that a significant change was introduced at Generation 9 (NEO-M9S, ZED-F9S, etc.) - a new Configuration Database was introduced and all configuration changes are made via the CFG-VALSET/GET/DEL messages rather than the earlier UBX-CFG messages. The UBX-CFG messages will continue to work for now but are deprecated and will be removed in future device generations. If you're using an older pre-Gen9 device (such as your NEO-M8U), the legacy UBX-CFG messages are still supported. See https://github.com/semuconsulting/pyubx2#configinterface for further details.

Depending on your needs, you may find the PyGPSClient GUI easier to use than the command line. It supports most of the common legacy UBX-CFG messages and a full range of the newer CFG-VALSET/GET/DEL messages. It also allows you to store user-defined command sequences that you may want to use repeatedly. See https://github.com/semuconsulting/PyGPSClient#userdefined for further details.

In terms of Save on Shutdown, if you're fairly new to UBX devices I would be inclined to steer clear of the UPD-SOS commands as ill-advised firmware updates can brick the device. I believe NEO-M8U devices have non-volatile ("battery-backed RAM" or BBR) storage so you should be able to use the CFG-CFG command to save or restore the current configuration. Not sure if this includes the specific configuration you're referring to - have a look at the NEO-M8U Interface Specification for further details, or raise the query on the u-blox support forum.

Hope this helps.