u-blox / ubxlib

Portable C libraries which provide APIs to build applications with u-blox products and services. Delivered as add-on to existing microcontroller and RTOS SDKs.
Apache License 2.0
312 stars 94 forks source link

Sending Chunks Instead of Large Buffer in uGnssMgaResponseSend() #274

Closed nautalert closed 2 months ago

nautalert commented 2 months ago

Trying to limit memory usage since the buffer required for offline data can be rather large. Is there a way to send it in smaller chunks instead of one large buffer?

RobMeades commented 2 months ago

Hi, and thanks for posting. The MGA data is just a literal stream of discrete UBX messages that the GNSS chip understands, so you can parse the MGA data to find the messages in it and then send each message down one at a time, or whatever you wish, to fit with your buffer limitations, just make sure that you send whole UBX messages.

See the function uUbxProtocolDecode() for how to do this. Note that the message returns the number of bytes in the body but the GNSS chip requires the entire message, with header (six bytes) and FCS (two bytes), so you will need to add 8 to the returned length.

https://github.com/u-blox/ubxlib/blob/2896142106fe596fb289748c68ba4afd8c1e717b/common/ubx_protocol/api/u_ubx_protocol.h#L154-L226

In other words:

// Assume that pBuffer is a pointer to your MGA data and bufferLength is the amount of data stored there

char *pMessageEnd = 0;
int32_t messageLength;

// Not bothering to give the function an output buffer or return the message class/ID: all we need to know
// is that there is a message (i.e. that the return value is non-negative) and get a pointer to the buffer just
// after that message
messageLength = uUbxProtocolDecode(pBuffer, bufferLength, NULL, NULL, NULL, 0,  &pMessageEnd));

// At this point messageLength is the length of the message body only
if (messageLength >= 0) {
    // Got something - add 8 to messageLength to make it the whole message length, not just the body
    messageLength += 8;
    printf("Found a UBX message of length %d bytes (including header and FCS) at location %p.\n", messageLength, pMessageEnd - messageLength);
} else if (messageLength == (int32_t) U_ERROR_COMMON_TIMEOUT) {
    printf("Partial UBX message found - more input data required.\n");
} else {
    printf("No UBX message found in input data.\n");
}

...in a loop of course, with pBuffer advanced and bufferLength reduced appropriately each time around.

RobMeades commented 2 months ago

@nautalert: does the above help you at all?

nautalert commented 2 months ago

So just sending the commands via uGnssMsgSend() is all we need to do?

RobMeades commented 2 months ago

Yes, once you have chopped-up the data downloaded from the MGA server into discrete (whole) UBX messages, send them all down (in order), one after the other, and the GNSS chip will act upon them. From memory each message is relatively short (maybe 50 or 100 bytes), which is a lot better than many kbytes,

nautalert commented 2 months ago

Yes, 84 bytes to be exact. :) Thanks for the info. Is this what is happening on the backend? Because this method seems to take longer.

RobMeades commented 2 months ago

Yes, kind of. uGnssMsgSend() will always check for an acknowledgment from the GNSS chip, each time.

uGnssMgaResponseSend() includes a flowControl parameter which allows you to chose how much acking you need from the GNSS chip:

The comment above uGnssMgaResponseSend() warns that you must send a complete MGA response to the GNSS chip but that is only because the GNSS chip does require complete UBX messages, so you can't just randomly chop the MGA response up. If you have parsed the MGA response into discrete (whole) UBX messages, you could use uGnssMgaResponseSend() to send down [a set of] those UBX messages with, say U_GNSS_MGA_FLOW_CONTROL_WAIT, if you think that would be faster than waiting for an ack for each one.

RobMeades commented 2 months ago

Sorry, no, I'm talking rubbish, uGnssMsgSend() does NOT wait for an ack at all, since it has no idea what you are sending (UBX message or NMEA message or whatever), so it should be like using uGnssMgaResponseSend() without even the 10 ms delay.

nautalert commented 2 months ago

Thanks. That appears to be work.

RobMeades commented 2 months ago

Great, thanks for letting us know.