NEO-M8U IMU data (UBX_ESF_RAW_MAX_LEN not correct)

[adamoskaranikas]

Subject of the issue

This is more of a question than an issue. I'm using a NEO-M8U at high nav rate (~30Hz) and its been really good. After diving in to it a bit more, I've began wondering if it's possible to extract its IMU data at rates higher than 30Hz. If this is possible, it'll allow me to remove an external IMU from my setup which would save quite a bit of cost.

Your workbench

The module is connected to a ESP32 WROOM32D via qwic i2C. It's normally powered by USB, but eventually I'll be powering everything out of a motorcycle can bus.

I was wondering if you guys have any experience with getting the IMU data out separately from the UBX message at different rates. Any help is greatly appreciated!

PS I know that I should probably be asking this on the UBLOX forum, but the response times there are atrocious and this is slightly time sensitive so I'm hoping I could pick your guys' brains about this idea.

[PaulZC]

Hi @adamoskaranikas ,

Let's walk this through...

The datasheet says that the M8U's Max navigation rate is 30Hz. But it also says that the max sensor measurement output rate is 100Hz.

Then in section 1.9: "For post-processing applications sensor data is available from messages UBX-ESF-MEAS and UBX-ESF-RAW (high rate). Each message includes the time of measurement."

So, if I'm reading this right, you should be able to request ESF-RAW at the full 100Hz.

My starting point would be Example5 which lets you access the ESF data using callbacks. The example doesn't include RAW but the functions are there:

setAutoESFRAWcallbackPtr will set up the callback.

There's a cryptic note in the M8 protocol description for UBX-ESF-RAW which says "Note that the rate selected in UBX-CFG-MSG is not respected. If a positive rate is selected then all raw measurements will be output.". So it sounds like you should get RAW at the full rate as soon as it is enabled?

More detail: " 29.5.6 Raw Sensor Data Output Some u-blox module products contain inertial sensors (IMU) that are directly connected to the GNSS and cannot be directly accessed from outside the module. The UBX-ESF-RAW message can be used to access raw measurements of these sensors. A variable number of data fields may be used in a single message and these can contain different types of measurements. The type of each measurement is specified in the dataType field. The possible data types are x, y and z-axis measurements on gyroscope or accelerometer and gyroscope temperature measurements as described in the ESF Measurement Data section. One UBX-ESF-RAW message can contain multiple samples from the same sensor. The user can separate and order these using the time tags attached to each of the measurements. The measurements are made at a fixed rate. The sampling rate or other sensor configuration options can not be changed. To turn on this feature the UBX-ESF-RAW message must be enabled using UBX-CFG-MSG. If nonzero rate is selected the message will be output but the selected rate does not otherwise have an influence at the rate of the messages. "

So, again, this suggests you cannot actually set the rate of the RAW messages. Once enabled, you get them at the maximum rate by default?

One thing you will need to change is this:

 myGNSS.setI2CpollingWait(5); //Allow checkUblox to poll I2C data every 5ms to keep up with the ESF RAW messages

The ESF-RAW message will be 68 bytes long. At 100Hz, that's 6800 bytes/sec. With I2C, that's approximately 70 kbits/sec. So you will probably find that 100kHz I2C runs out of steam, especially if you have other messages enabled. You're going to need to use 400kHz.

Please give it a try and see what happens.

I don't have time to test this for you. You will need to do all of the experimenting yourself. But I'm happy to leave this issue open until you've got it working. Let me know if you need more pointers.

Best wishes, Paul

[adamoskaranikas]

Hello @PaulZC ,

Thanks a ton for the pointers, for me this is a great starting point, I was aware of some of the points you brought up, but it's nice to have the validation from someone with a bit more experience in the area. I'll start doing some tests and will update on the issue, as I feel like I'm not the only one that has thought of implementing this functionality. That way there can be a starting point for others.

[PaulZC]

Hi Adam,

Please let us know if you need more help with this - otherwise can you please close this issue?

Very best wishes, Paul

[NicoZobernig]

Hi @PaulZC,

I found this issue just now, looking to basically do the same as Adam. To summarize, I would like to get the raw IMU measurements out (UBX_ESF_RAW), hoping that these are available even before getting to a proper calibration (fix type 4).

I followed the same procedure as in the ESF callback examples (which work for me) for the ESF_RAW message, but unfortunately to no avail.

Do you have any pointers as to how I could get raw IMU readings via that msg. From the documentation and your post above, I understand the module should be able to do that...

Best, Nico

[PaulZC]

Hi Nico,

Are you using I2C to communicate with the NEO? I'm wondering if 100kHz I2C is the bottleneck, preventing the messages from being generated?

How often are you calling checkUblox? You will need to call it almost continuously to prevent the buffer from filling up.

Have you called setI2CpollingWait and set the polling wait to (say) 5ms?

I just tried enabling ESF RAW on USB using u-center and it's working:

Wow... It is generating a LOT of data... It appears to be generating 0x23C = 572 bytes in each message. Every ~50ms...

OK. That's about 91kBits/sec. There's no way 100kHz I2C will be able to keep up... You really need 400kHz I2C as a minimum.

If you switch to Serial - the library supports both - you're going to have to change the UART1 baud rate to 460800 to stand a chance of keeping up...

SPI might be a smart move. The library supports that too.

That's all the time I can spare for now. Please dig into this yourself and let me know what you find. Please reach out again next week if you're still having problems.

Best wishes, Paul

[PaulZC]

Hi @NicoZobernig @adamoskaranikas ,

Hang on! The ESF RAW messages are much bigger than I was expecting...

When I added ESF RAW to the library, I assumed each message would contain one 'set' of raw sensor measurements. But the NEO is sending way more than that. The messages above contain 564 data bytes. Subtract 4 for the reserved1 field, and what follows is 70 sets of 8-byte data. There are 7 sensor channels: 3 x gyro; 3 x accel; 1 x gyro temperature. So the NEO is sending the data in groups of 10 readings...

OK. We need to change this line:

https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library/blob/20e265db38e9ae5f131a097fb75ecc62a785d7c1/src/u-blox_structs.h#L2260

to something like:

#define DEF_ESF_RAW_REPEATS 10
const uint16_t UBX_ESF_RAW_MAX_LEN = 4 + (8 * DEF_NUM_SENS * DEF_ESF_RAW_REPEATS);

Humm. Yes. That might explain why you've not been getting any data!

Ah. We need to make a change here too:

https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library/blob/20e265db38e9ae5f131a097fb75ecc62a785d7c1/src/SparkFun_u-blox_GNSS_Arduino_Library.cpp#L4249

to something like:

for (uint16_t i = 0; (i < (DEF_NUM_SENS * DEF_ESF_RAW_REPEATS)) && ((i * 8) < (msg->len - 4)); i++)

Please try making these changes and let me know what happens. (I don't have time to do it myself. Sorry!)

Thank you, Paul

[PaulZC]

The more I think about this, the more weaknesses I see…. This line needs to change:

https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library/blob/20e265db38e9ae5f131a097fb75ecc62a785d7c1/src/u-blox_structs.h#L2279

[PaulZC]

This is problematic too. It can only access the first sensor block. It is unaware of blocks 2-10….

https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library/blob/20e265db38e9ae5f131a097fb75ecc62a785d7c1/src/SparkFun_u-blox_GNSS_Arduino_Library.cpp#L18058

[NicoZobernig]

Hi @PaulZC, back in front of my PC, will start to look into this now!

I did the setup correctly I believe, ESP32 with 1kHz tickrate, I2C running at 400kHz, called

neo.setI2CpollingWait(5);
neo.setAutoESFRAW(true);
if (neo.setAutoESFRAWcallbackPtr(&esf_raw_cb) == true) {
    ESP_LOGI(neoTag, "Registered ESF RAW callback");
}

and call checkUblox() in every loop of a task running with a delay of 10 ticks, so should be at around 100 Hz. And then

if (neo.packetUBXESFRAW->automaticFlags.flags.bits.callbackCopyValid == true) {
    ESP_LOGI(neoTag, "got RAW data");
    uint32_t stamp = neo.packetUBXESFRAW->data.data[0].sTag;
    uint32_t sensor_type = neo.packetUBXESFRAW->data.data[0].data.bits.dataType;
    ESP_LOGI(neoTag, "t: %u - id: %u", stamp, sensor_type);
}

to check if I'm getting something.

After including the proposed changes (all besides getRawSensorMeasurement since there is no need yet), I am finally getting something!

I (942) NEO: Registered ESF RAW callback                                                                                                                                                                           
I (991) NEO: got RAW data                                                                                                                                                                                          
I (991) NEO: t: 10721806 - id: 16                                                                                                                                                                                  
I (1002) NEO: got RAW data                                                                                                                                                                                         
I (1003) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1003) NEO: got RAW data                                                                                                                                                                                         
I (1003) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1006) NEO: got RAW data                                                                                                                                                                                         
I (1010) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1014) NEO: got RAW data                                                                                                                                                                                         
I (1017) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1031) NEO: got RAW data                                                                                                                                                                                         
I (1031) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1031) NEO: got RAW data                                                                                                                                                                                         
I (1032) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1036) NEO: got RAW data                                                                                                                                                                                         
I (1040) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1044) NEO: got RAW data                                                                                                                                                                                         
I (1047) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1063) NEO: got RAW data                                                                                                                                                                                         
I (1063) NEO: t: 10721806 - id: 16                                                                                                                                                                                 
I (1093) NEO: got RAW data                                                                                                                                                                                         
I (1093) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1104) NEO: got RAW data                                                                                                                                                                                         
I (1105) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1112) NEO: got RAW data                                                                                                                                                                                         
I (1112) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1116) NEO: got RAW data                                                                                                                                                                                         
I (1116) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1118) NEO: got RAW data                                                                                                                                                                                         
I (1121) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1126) NEO: got RAW data                                                                                                                                                                                         
I (1129) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1133) NEO: got RAW data                                                                                                                                                                                         
I (1136) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1141) NEO: got RAW data                                                                                                                                                                                         
I (1144) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1159) NEO: got RAW data                                                                                                                                                                                         
I (1159) NEO: t: 10724373 - id: 16                                                                                                                                                                                 
I (1190) NEO: got RAW data                                                                                                                                                                                         
I (1190) NEO: t: 10726942 - id: 16                                                                                                                                                                                 
I (1201) NEO: got RAW data                                                                                                                                                                                         
I (1201) NEO: t: 10726942 - id: 16                                                                                                                                                                                 
I (1202) NEO: got RAW data                                                                                                                                                                                         
I (1202) NEO: t: 10726942 - id: 16
I (1205) NEO: got RAW data
I (1208) NEO: t: 10726942 - id: 16
I (1213) NEO: got RAW data
I (1216) NEO: t: 10726942 - id: 16
I (1231) NEO: got RAW data
I (1231) NEO: t: 10726942 - id: 16
I (1232) NEO: got RAW data
I (1232) NEO: t: 10726942 - id: 16
I (1238) NEO: got RAW data
I (1241) NEO: t: 10726942 - id: 16
I (1254) NEO: got RAW data
I (1254) NEO: t: 10726942 - id: 16
I (1255) NEO: got RAW data
I (1256) NEO: t: 10726942 - id: 16

I am, however, still a little confused about what exactly I am getting here... I will continue playing around with it.

[NicoZobernig]

Also, what is the purpose of checkCallbacks() ? Do I need to call that every loop as well? Per my understanding, the packets are being populated anyway, so I can just access the data directly?

However, when also calling it at the end of the task loop, I am getting output that seems more correct in a way:

I (939) NEO: Registered ESF RAW callback                                                                                                                                                                           
I (983) NEO: got RAW data                                                                                                                                                                                          
I (983) NEO: t: 13472525 - id: 16                                                                                                                                                                                  
I (983) ESF RAW CB: t: 13472525                                                                                                                                                                                    
I (1083) NEO: got RAW data                                                                                                                                                                                         
I (1083) NEO: t: 13475093 - id: 16                                                                                                                                                                                 
I (1083) ESF RAW CB: t: 13475093                                                                                                                                                                                   
I (1191) NEO: got RAW data                                                                                                                                                                                         
I (1191) NEO: t: 13477662 - id: 16                                                                                                                                                                                 
I (1191) ESF RAW CB: t: 13477662                                                                                                                                                                                   
I (1293) NEO: got RAW data                                                                                                                                                                                         
I (1293) NEO: t: 13480206 - id: 16                                                                                                                                                                                 
I (1293) ESF RAW CB: t: 13480206                                                                                                                                                                                   
I (1389) NEO: got RAW data                                                                                                                                                                                         
I (1389) NEO: t: 13482775 - id: 16                                                                                                                                                                                 
I (1389) ESF RAW CB: t: 13482775                                                                                                                                                                                   
I (1491) NEO: got RAW data                                                                                                                                                                                         
I (1491) NEO: t: 13485325 - id: 16                                                                                                                                                                                 
I (1491) ESF RAW CB: t: 13485325                                                                                                                                                                                   
I (1588) NEO: got RAW data                                                                                                                                                                                         
I (1588) NEO: t: 13487895 - id: 16                                                                                                                                                                                 
I (1589) ESF RAW CB: t: 13487895
I (1685) NEO: got RAW data
I (1685) NEO: t: 13490457 - id: 16
I (1685) ESF RAW CB: t: 13490457

Only getting a specific timestamp once. The rate seems quite slow though, definitely not the 100Hz I hoped for yet.

[NicoZobernig]

Hi @PaulZC!

I got something going, manage to read it all out now :)

The code is more or less calling checkUblox() and then

if (neo.packetUBXESFRAW->automaticFlags.flags.bits.callbackCopyValid == true) {
  uint32_t ticks = xTaskGetTickCount();
  ESP_LOGI(neoTag, "-----------------------------------");
  ESP_LOGI(neoTag, "got RAW data [t: %u - ticks: %u]", neo.packetUBXESFRAW->data.data[0].sTag, ticks);

  for (int i=0; i < DEF_NUM_SENS * DEF_ESF_RAW_REPEATS; i++) {
    uint32_t data_type = neo.packetUBXESFRAW->data.data[i].data.bits.dataType;
    if (data_type == 13) {
      ticks = xTaskGetTickCount();
      int32_t y_gyro_int = parseSensorData(neo.packetUBXESFRAW->data.data[i].data.all);
      double y_gyro = (double)y_gyro_int / 4096.0;
      ESP_LOGI(neoTag, "[t: %u - ticks: %u] y gyro: %f", neo.packetUBXESFRAW->data.data[i].sTag, ticks, y_gyro);
    }
  }
  neo.packetUBXESFRAW->automaticFlags.flags.bits.callbackCopyValid = false;
}

inside the main loop of the task.

The code in the callback is not getting executed if I don't call checkCallbacks(), which I don't quite get yet, maybe you have some pointers for me. Since the data is available within the main loop, I don't see a specific need for the current state, just seems a litlle weird to me.

Anyway, the output looks like this:

I (8279) NEO: -----------------------------------
I (8279) NEO: got RAW data [t: 3999518 - ticks: 7941]
I (8279) NEO: [t: 3999518 - ticks: 7941] y gyro: 6.364258
I (8285) NEO: [t: 3999775 - ticks: 7947] y gyro: 6.836914
I (8291) NEO: [t: 4000032 - ticks: 7953] y gyro: 7.644775
I (8297) NEO: [t: 4000290 - ticks: 7959] y gyro: 8.216309
I (8303) NEO: [t: 4000546 - ticks: 7965] y gyro: 8.544189
I (8309) NEO: [t: 4000802 - ticks: 7971] y gyro: 9.474121
I (8315) NEO: [t: 4001034 - ticks: 7977] y gyro: 11.295654
I (8321) NEO: [t: 4001291 - ticks: 7983] y gyro: 13.635742
I (8328) NEO: [t: 4001547 - ticks: 7990] y gyro: 16.303223
I (8334) NEO: [t: 4001804 - ticks: 7996] y gyro: 19.557861
I (8429) NEO: -----------------------------------
I (8430) NEO: got RAW data [t: 4004630 - ticks: 8091]
I (8430) NEO: [t: 4004630 - ticks: 8092] y gyro: 12.256104
I (8436) NEO: [t: 4004887 - ticks: 8098] y gyro: 7.644775
I (8442) NEO: [t: 4005144 - ticks: 8104] y gyro: 1.303223
I (8448) NEO: [t: 4005402 - ticks: 8110] y gyro: -5.457520
I (8454) NEO: [t: 4005662 - ticks: 8116] y gyro: -10.937744
I (8460) NEO: [t: 4005919 - ticks: 8122] y gyro: -14.169434
I (8466) NEO: [t: 4006172 - ticks: 8128] y gyro: -15.625244
I (8473) NEO: [t: 4006429 - ticks: 8135] y gyro: -15.670898
I (8479) NEO: [t: 4006686 - ticks: 8141] y gyro: -14.588623
I (8485) NEO: [t: 4006942 - ticks: 8147] y gyro: -12.683105

which seems correct.

The one thing that's bothering me is that the data is only available at around 10 Hz, although we do get around 10 sensor readings per batch (ergo 100Hz). But since new data is only available at this lower rate, I don't see the point in having old data. I believe this behavior is coming from the NEO though, and not the library, is that correct?

[PaulZC]

Hi Nico (@NicoZobernig ),

Great job - thanks for digging into this.

The callback only gets called if you call checkCallbacks(). checkCallbacks() will call any/all callbacks that have been defined and which have fresh valid data from the GNSS module. But, correct, you don't actually need to call checkCallbacks(). You can access the data manually, from inside the main loop. There is an example showing how to do that.

When I wrote the ESF RAW "auto" code, I was expecting the module to output the RAW data one 'set' of measurements at a time, like it does when you poll a single set. I suspect the module is sending 10 at a time to make the interface more efficient. The overhead is 8 bytes per packet/message. In summary, it is outputting data at 100Hz. You do get all the data, and each reading is timestamped. It is just arriving in sets or groups of 10 readings, not one at a time.

You need to be a little bit careful when manipulating the sensor data. It is 24-bit signed (two's complement). There is a forum post here that might help.

OK. I've got some work to do here. As a minimum, I need to add an extra field to UBX_ESF_RAW_data_t which will tell you how many sets of data it contains. We know that the NEO-M8U is sending sets of 10 readings. I suspect the ZED-F9R will also send sets of 10, but I need to check. What we need to end up with is a system that works for individual readings (polled one at a time) and sets of readings (streamed at the full 100Hz) on all modules that support ESF RAW.

I won't be able to get to this straight away. I'm working on a different project at the moment. But I will hopefully get to it in a few days. Thank you for your patience. The other project is to do with logging data at high speed, and logging ESF RAW data would a great example for that, so I might use that as an excuse to work on this sooner! ;-)

Very best wishes, Paul

[PaulZC]

Hi @NicoZobernig @adamoskaranikas ,

OK... I believe this is fully resolved in v2.2.14. Please see the release notes for more details. Please try the new examples and let me know if you see any more weirdness.

Thank you for your patience - and for reporting this issue. It is good to have it fixed.

I'm going to close this issue as I believe it is - now - fully resolved, but please re-open if necessary.

Enjoy! Paul

[shecker]

Hi @PaulZC Thanks for updating the library. I can confirm that both examples for obtaining ESF_RAW (callback and in-loop) works for the NEO-M8U.

However, when replacing the NEO-M8U with a ZED-F9K (automotive version of ZED-F9R) we cannot receive the raw sensor measurements.

Is this an issue with the module or the library? From my understanding, there is no difference between F9R and F9K apart from the automotive rating (temperature, ruggedness etc.) Can you confirm this understanding and if so do you know why the F9K cannot report raw sensor measurements?

Thanks!

[PaulZC]

Hi Simon (@shecker ),

What version of the HPS firmware is your F9K module running? I'm wondering if this is linked to the UBX-CFG-MSG being deprecated. Or, maybe the F9K outputs the RAW data differently to the F9R?

Some things to try:

Please enable the debug messages with myGNSS.enableDebugging();. Look closely at the output. Do you see any NACK's (Negative ACKnowledgements)? If so, please post the output here.

Do you see any ESF-RAW data arriving: Class 0x10 ID 0x03? If so, please post the data here.

Please also try v3 of the GNSS Library: https://github.com/sparkfun/SparkFun_u-blox_GNSS_v3

Best wishes, Paul

[PaulZC]

Please also scroll back to this comment:

https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library/issues/133#issuecomment-1235667721

It would be helpful if you could do the exact same thing with the F9K: enable ESF-RAW on USB using u-center; copy and paste the HEX packet here so I can take a look. I'd like to check the F9K is outputting the same amount of data as the F9R.

[shecker]

Hi @PaulZC Thanks for the quick reply. Indeed the ESP-RAW messages are not being published in u-center.

We checked the firmware of our ZED-F9K module and this is version 1.00. Unfortunately there is no firmware update available on the official ZED-F9K product support page. However, there is a newer firmware HPS1.30 available for the ZED-F9R. Do you know if this is compatible and we can flash this? We would like to avoid bricking the device. Thanks!

[PaulZC]

Hi Simon,

I'm afraid I've got zero experience with the F9K... The best way forward is to create an account at u-blox.com, go to the support portal, and open a support case. (I see you've already asked a public question about this.)

Best wishes, Paul

[jinhyeongRepo]

Hi @PaulZC

The updated example was very helpful.

I used the code of "SparkFun_u-blox_GNSS_Arduino_Library/examples/Callbacks/CallbackExample12_ESF_MEAS_In_Loop/". However, we found one problem with running this file. The acceleration is updated normally, but the gyro value is very slow to update. I used ZED-F9R and ESP32, have you had this problem before?

Thank you.

[PaulZC]

Hello @jinhyeongRepo ,

I do all of my work from my desk. It is difficult for me to put the F9R into a car - to perform the calibration.

Is the gyro slow to update both before and after the calibration?

How are you testing the gyro? In a vehicle? The gyro will only give a non-zero value when the sensor's angular velocity is changing.

Best wishes, Paul

[jinhyeongRepo]

Hello @PaulZC

Thank you for your answer. I'm testing in two ways using u-center and esp32. In both cases, the calibration process was not conducted. For the gyro test, the test was conducted by comparing the value measured by u-center with the value measured through esp32. The sensor was hand-shaken to see the change in gyro values.

1. In u-center, view -> messages view -> UBX -> MEAS -> graph or RAW. I checked that the value of the IMU sensor can be known without the calibration process.

2. In esp32, the values have been read using "myGNSS.packetUBXESFMEAS". However, in this case, the gyro value was updated slowly, unlike when using u-center. especially the x-axis gyro value is being updated very slowly.

   #include <Wire.h> //Needed for I2C to GPS
   #include <SparkFun_u-blox_GNSS_Arduino_Library.h> //http://librarymanager/All#SparkFun_u-blox_GNSS
   SFE_UBLOX_GNSS myGNSS;
   
   void printESFMEASdata(UBX_ESF_MEAS_data_t *ubxDataStruct)
   {
   Serial.println(F("Hey! The ESF RAW callback has been called!"));
   }
   
   float Zgyr = 0;
   float Temp = 0;
   float Ygyr = 0;
   float Xgyr = 0;
   float Xacc = 0;
   float Yacc = 0;
   float Zacc = 0;
   
   void setup()
   {
   Serial.begin(230400); // <--- Use >> 100k baud (see notes above)
   
   while (!Serial); //Wait for user to open terminal
   Serial.println(F("SparkFun u-blox Example"));
   
   Wire.begin();
   Wire.setClock(400000); // <-- Use 400kHz I2C (ESSENTIAL)
   
   // myGNSS.enableDebugging(); // Uncomment this line to enable debug messages on Serial
   
   if (myGNSS.begin() == false) //Connect to the u-blox module using Wire port
   {
     Serial.println(F("u-blox GNSS not detected at default I2C address. Please check wiring. Freezing."));
     while (1);
   }
   
   myGNSS.setI2COutput(COM_TYPE_UBX); //Set the I2C port to output UBX only (turn off NMEA noise)
   myGNSS.saveConfigSelective(VAL_CFG_SUBSEC_IOPORT); //Save (only) the communications port settings to flash and BBR
   
   myGNSS.setI2CpollingWait(5); //Allow checkUblox to poll I2C data every 5ms to keep up with the ESF RAW messages
   
   if (myGNSS.setAutoESFMEAScallbackPtr(&printESFMEASdata) == true) // Enable automatic ESF RAW messages with callback to printESFRAWdata
     Serial.println(F("setAutoESFRAWcallback successful"));
   }
   
   void loop()
   {
   myGNSS.checkUblox(); // Check for the arrival of new data and process it.
   
   // Check if new ESF RAW data has arrived:
   // If myGNSS.packetUBXESFMEAS->automaticFlags.flags.bits.callbackCopyValid is true, it indicates new ESF RAW data has been received and has been copied.
   // automaticFlags.flags.bits.callbackCopyValid will be cleared automatically when the callback is called.
   if (myGNSS.packetUBXESFMEAS->automaticFlags.flags.bits.callbackCopyValid == true)
   {
     // But, we can manually clear the callback flag too. This will prevent the callback from being called!
     myGNSS.packetUBXESFMEAS->automaticFlags.flags.bits.callbackCopyValid = false; // Comment this line if you still want the callback to be called
   
     uint32_t sTag = 0xFFFFFFFF; // Sensor time tag
   
     // Only print the first seven sensor readings (on the NEO-M8U)
     for (uint8_t i = 0; (i < 7); i++)
     {
       // Print the sensor data type
       // From the M8 interface description:
       //   0: None
       // 1-4: Reserved
       //   5: z-axis gyroscope angular rate deg/s * 2^-12 signed
       //   6: front-left wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
       //   7: front-right wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
       //   8: rear-left wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
       //   9: rear-right wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
       //  10: speed ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
       //  11: speed m/s * 1e-3 signed
       //  12: gyroscope temperature deg Celsius * 1e-2 signed
       //  13: y-axis gyroscope angular rate deg/s * 2^-12 signed
       //  14: x-axis gyroscope angular rate deg/s * 2^-12 signed
       //  16: x-axis accelerometer specific force m/s^2 * 2^-10 signed
       //  17: y-axis accelerometer specific force m/s^2 * 2^-10 signed
       //  18: z-axis accelerometer specific force m/s^2 * 2^-10 signed
       Serial.print("Xacc: ");
       Serial.print(Xacc);
       Serial.print("\t");
       Serial.print("Yacc: ");
       Serial.print(Yacc);
       Serial.print("\t");
       Serial.print("Zacc: ");
       Serial.print(Zacc);
       Serial.print("\t");
       Serial.print("Xgyr: ");
       Serial.print(Xgyr);
       Serial.print("\t");
       Serial.print("Ygyr: ");
       Serial.print(Ygyr);
       Serial.print("\t");
       Serial.print("Zgyr: ");
       Serial.print(Zgyr);
       Serial.print("\t");
       Serial.print("Temp: ");
       Serial.println(Temp);
   
       // Gyro data
       if ((myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 5) || (myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 13) || (myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 14))
       {
         union
         {
           int32_t signed32;
           uint32_t unsigned32;
         } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
         // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
         signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
         float rate =  signedUnsigned.signed32; // Extract the signed data. Convert to float
         rate /= 256.0; // Divide by 256 to undo the shift
         rate *= 0.000244140625; // Convert from deg/s * 2^-12 to deg/s
         // Serial.println(rate);     
         if(myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 5) Zgyr = rate;
         else if(myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 13) Ygyr = rate;
         else if(myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 14) Xgyr = rate;
       }
       // Accelerometer data
       else if ((myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 16) || (myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 17) || (myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 18))
       {
         union
         {
           int32_t signed32;
           uint32_t unsigned32;
         } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
         // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
         signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
         float force =  signedUnsigned.signed32; // Extract the signed data. Convert to float
         force /= 256.0; // Divide by 256 to undo the shift
         force *= 0.0009765625; // Convert from m/s^2 * 2^-10 to m/s^2
         // Serial.println(force);  
         if(myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 16) Xacc = force;
         else if(myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 17) Yacc = force;
         else if(myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 18) Zacc = force;   
       }
       // Gyro Temperature
       else if (myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataType == 12)
       {
         union
         {
           int32_t signed32;
           uint32_t unsigned32;
         } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
         // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
         signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->callbackData->data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
         float temperature =  signedUnsigned.signed32; // Extract the signed data. Convert to float
         temperature /= 256.0; // Divide by 256 to undo the shift
         temperature *= 0.01; // Convert from C * 1e-2 to C
         // Serial.println(temperature);   
         Temp = temperature;  
       }
     }
   }
   
   myGNSS.checkCallbacks(); // Check if any callbacks are waiting to be processed. There will not be any in this example, unless you commented the line above  
   }

Thanks!

[PaulZC]

Hello @jinhyeongRepo ,

Thank you. I had not used the "graph" option in u-center before. That is very useful!

OK. I see the same issue. I see the Time and 3*Accel arriving rapidly:

The speed ticks arrive much less often. The gyro data - almost never...

I suspect this is to do with the I2C bandwidth. We are using 400kHz, but maybe it is not fast enough?

I will try SPI next... Paul

[PaulZC]

Very curious...

This is from the ESF Callback example - modified so it uses SPI:

Sensor Type 16, 17 and 18 are the accelerometers. Type 10 is the speed tick. Again, almost zero gyro data.

If I change the code so it prints only gyro data, I see almost zero data:

Investigating...

[PaulZC]

Ah. OK. I understand what is happening!

Looking at the ESF MEAS data in u-center:

Notice how six messages arrive all together at 12:05:58.589:

The data highlighted in green is accelerometer data (16, 17 and 18) (0x10, 0x11 and 0x12).

The data highlighted in blue is the gyro data (14, 13, 5 and 12) (0x0E, 0x0D, 0x05 and 0x0C).

If the data is all being added to the I2C - or SPI - buffer at the same time, then the library code will only save the first message and will ignore the others as it has nowhere to store them. (The code does not allow the callback copy to be overwritten.)

OK. To be able to record the accelerometer and gyro data, I need to create additional storage. Possibly some kind of circular buffer (ring buffer) like we do for the MGA messages:

https://github.com/sparkfun/SparkFun_u-blox_GNSS_Arduino_Library/blob/2ae72895a1194c77e0628d50ec51937016ac7466/src/u-blox_structs.h#L2385-L2391

OK. I am going to create an issue (feature request) for this: #179

It will take me some time to implement this. Thanks again for reporting.

Best wishes, Paul

[PaulZC]

It's a nasty hack, but you can access the gyro data by reading data instead of callbackData. data does get overwritten and - purely by chance has the gyro data in it (but only because the gyro data arrives last):

Here is the modified code:

void loop()
{
  myGNSS.checkUblox(); // Check for the arrival of new data and process it.

  // Check if new ESF MEAS data has arrived:
  // If myGNSS.packetUBXESFMEAS->automaticFlags.flags.bits.callbackCopyValid is true, it indicates new ESF MEAS data has been received and has been copied.
  // automaticFlags.flags.bits.callbackCopyValid will be cleared automatically when the callback is called.

  if (myGNSS.packetUBXESFMEAS->automaticFlags.flags.bits.callbackCopyValid == true)
  {
    // But, we can manually clear the callback flag too. This will prevent the callback from being called!
    myGNSS.packetUBXESFMEAS->automaticFlags.flags.bits.callbackCopyValid = false; // Comment this line if you still want the callback to be called

    // Print the timeTag
    Serial.print(F("Time:        "));
    Serial.println(myGNSS.packetUBXESFMEAS->data.timeTag);

    //Serial.print(F("Num Meas:    "));
    //Serial.println(myGNSS.packetUBXESFMEAS->data.flags.bits.numMeas);

    // myGNSS.packetUBXESFMEAS->data.flags.bits.numMeas indicates how many sensor groups the UBX_ESF_MEAS_data_t contains.
    for (uint8_t i = 0; i < myGNSS.packetUBXESFMEAS->data.flags.bits.numMeas; i++)
    {  
      // Print the sensor data type
      // From the M8 interface description:
      //   0: None
      // 1-4: Reserved
      //   5: z-axis gyroscope angular rate deg/s * 2^-12 signed
      //   6: front-left wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
      //   7: front-right wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
      //   8: rear-left wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
      //   9: rear-right wheel ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
      //  10: speed ticks: Bits 0-22: unsigned tick value. Bit 23: direction indicator (0=forward, 1=backward)
      //  11: speed m/s * 1e-3 signed
      //  12: gyroscope temperature deg Celsius * 1e-2 signed
      //  13: y-axis gyroscope angular rate deg/s * 2^-12 signed
      //  14: x-axis gyroscope angular rate deg/s * 2^-12 signed
      //  16: x-axis accelerometer specific force m/s^2 * 2^-10 signed
      //  17: y-axis accelerometer specific force m/s^2 * 2^-10 signed
      //  18: z-axis accelerometer specific force m/s^2 * 2^-10 signed
      switch (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType)
      {
        case 5:
          Serial.print(F("Z Gyro:      "));
          break;
        case 6:
          Serial.print(F("Front Left:  "));
          break;
        case 7:
          Serial.print(F("Front Right: "));
          break;
        case 8:
          Serial.print(F("Rear Left:   "));
          break;
        case 9:
          Serial.print(F("Rear Right:  "));
          break;
        case 10:
          Serial.print(F("Speed Ticks: "));
          break;
        case 11:
          Serial.print(F("Speed:       "));
          break;
        case 12:
          Serial.print(F("Temp:        "));
          break;
        case 13:
          Serial.print(F("Y Gyro:      "));
          break;
        case 14:
          Serial.print(F("X Gyro:      "));
          break;
        case 16:
          Serial.print(F("X Accel:     "));
          break;
        case 17:
          Serial.print(F("Y Accel:     "));
          break;
        case 18:
          Serial.print(F("Z Accel:     "));
          break;
        default:
          break;
      }

      // Tick data
      if ((myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType >= 6) && (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType <= 10))
      {
        if ((myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataField & (1 << 23)) > 0)
          Serial.print(F("-")); // Backward
        else
          Serial.print(F("+")); // Forward
        Serial.println(myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataField & 0x007FFFFF);
      }
      // Speed
      else if (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 11)
      {
        union
        {
          int32_t signed32;
          uint32_t unsigned32;
        } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
        // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
        signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
        float speed =  signedUnsigned.signed32; // Extract the signed data. Convert to float
        speed /= 256.0; // Divide by 256 to undo the shift
        speed *= 0.001; // Convert from m/s * 1e-3 to m/s
        Serial.println(speed, 3);     
      }
      // Gyro data
      else if ((myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 5) || (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 13) || (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 14))
      {
        union
        {
          int32_t signed32;
          uint32_t unsigned32;
        } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
        // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
        signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
        float rate =  signedUnsigned.signed32; // Extract the signed data. Convert to float
        rate /= 256.0; // Divide by 256 to undo the shift
        rate *= 0.000244140625; // Convert from deg/s * 2^-12 to deg/s
        Serial.println(rate);     
      }
      // Accelerometer data
      else if ((myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 16) || (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 17) || (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 18))
      {
        union
        {
          int32_t signed32;
          uint32_t unsigned32;
        } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
        // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
        signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
        float force =  signedUnsigned.signed32; // Extract the signed data. Convert to float
        force /= 256.0; // Divide by 256 to undo the shift
        force *= 0.0009765625; // Convert from m/s^2 * 2^-10 to m/s^2
        //Serial.println(force);     
      }
      // Gyro Temperature
      else if (myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataType == 12)
      {
        union
        {
          int32_t signed32;
          uint32_t unsigned32;
        } signedUnsigned; // Avoid any ambiguity casting uint32_t to int32_t
        // The dataField is 24-bit signed, stored in the 24 LSBs of a uint32_t
        signedUnsigned.unsigned32 = myGNSS.packetUBXESFMEAS->data.data[i].data.bits.dataField  << 8; // Shift left by 8 bits to correctly align the data
        float temperature =  signedUnsigned.signed32; // Extract the signed data. Convert to float
        temperature /= 256.0; // Divide by 256 to undo the shift
        temperature *= 0.01; // Convert from C * 1e-2 to C
        Serial.println(temperature);     
      }
    }
  }

  myGNSS.checkCallbacks(); // Check if any callbacks are waiting to be processed. There will not be any in this example, unless you commented the line above  
}

[jinhyeongRepo]

hello @PaulZC

Thank you for your help. I've been able to understand what's causing the issue thanks to you. I'm going to use an external sensor for now, and if everything works out, I'll update my project later.

Thanks again. jinhyeong

[PaulZC]

Or use ESF RAW?

Best wishes, Paul

[jinhyeongRepo]

Thank you for your suggestion to solve the problem.

1. "SparkFun_u-blox_GNSS_Arduino_Library/examples/Callbacks/CallbackExample11_ESF_RAW_In_Loop/"
2. "SparkFun_u-blox_GNSS_Arduino_Library/examples/Callbacks/CallbackExample10_ESF_RAW/CallbackExample10_ESF_RAW.ino" In my case, when I ran these two examples, I couldn't read the value of the sensor. Only the setup part is executed and the value is not updated in the callback part of the loop.

On the other hand, "SparkFun_u-blox_GNSS_Arduino_Library/examples/CallbackExample12_ESF_MEAS_In_Loop/CallbackExample12_ESF_MEAS_In_Loop/CallbackExample12_ESF_MEAS_In_Loop.ino", which uses MEAS instead of RAW, works well. However, in this case, the problem I mentioned earlier occurred.

It's late now, so I think I have to work on it tomorrow. Thank you !!

[PaulZC]

Hi jinhyeong (@jinhyeongRepo ),

That's very strange... Example10 and Example11 work OK for me:

Please check what version of firmware your module is running. In u-center, open UBX-MON-VER:

Perhaps you need to upgrade?

Best wishes, Paul

[jinhyeongRepo]

Hi @PaulZC

Thank you. As you said, after updating the firmware the values are coming in normally!! Now I can do other things. :)

Thank you!! jinhyeong