The obtained data is lost.

[zytao-01]

Hello, when I successfully connected to MySQL and obtained data, I found that the data was not complete. For example, I had 1024 bytes of data, but only 250 bytes of data were obtained. How should I solve this problem?

[ChuckBell]

Hi. Yes, there is a hard limit for the amount of data you can retrieve. It is also possible you're exceeding the number of columns limit of 32 columns. The code is currently setup for 256 chars. You can change that by altering the code, but you should consider that move carefully because your board may not have enough memory available. That is, we must read an entire row into memory. One question I would ask is why you need that much data. You should consider carefully what you need to retrieve and retrieve only the columns you need and nothing more. Reduce your dependence on the entire row - these small devices and the library are not robust enough to handle "normal" SELECT queries of large (relatively) size data. That said, if you do need all 1024 bytes, consider retrieving the data in chunks (several cols at a time with multiple queries). If you can post a sample of your data, I may be able to make some suggestions.

[zytao-01]

I'm sorry, I don't know where in the code to make the change, is it in this place? int MySQL_Packet::read_lcb_int(int offset) { int len_size = 0; int size = 0; int value = 0; if (!buffer) return -1; len_size = buffer[offset]; if (len_size < 252) { return buffer[offset]; } else if (len_size == 252) { len_size = 2; } else if (len_size == 253) { len_size = 3; } else { len_size = 8; } int shifter = (len_size-1) * 8; for (int i = len_size; i > 0; i--) { value += (buffer[offset+i] << shifter); shifter -= 8; } return value; } If so, how do you change it? If not, please point to the right place, thank you. On the memory side, there is still enough memory to receive such long data.

[ChuckBell]

Ok. Good. I would start by debugging the MySQL_Packet::read_packet() and MySQL_Packet::wait_for_bytes() methods to see how many bytes it is reading and whether some changes need to be made to either wait longer or read more data. It should be reading the entire stream (1024). I think this is where things are going wrong.

On Feb 22, 2023, at 8:39 PM, zytao-01 @.***> wrote:

I'm sorry, I don't know where in the code to make the change, is it in this place? int MySQL_Packet::read_lcb_int(int offset) { int len_size = 0; int size = 0; int value = 0; if (!buffer) return -1; len_size = buffer[offset]; if (len_size < 252) { return buffer[offset]; } else if (len_size == 252) { len_size = 2; } else if (len_size == 253) { len_size = 3; } else { len_size = 8; } int shifter = (len_size-1) * 8; for (int i = len_size; i > 0; i--) { value += (buffer[offset+i] << shifter); shifter -= 8; } return value; } If so, how do you change it? If not, please point to the right place, thank you. On the memory side, there is still enough memory to receive such long data.

— Reply to this email directly, view it on GitHub https://github.com/ChuckBell/MySQL_Connector_Arduino/issues/204#issuecomment-1441117051, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6SHYHDQYMKVUSYURXUGLLWY25W7ANCNFSM6AAAAAAVBLEYMI. You are receiving this because you commented.

[zytao-01]

Sorry, I still don't know how to make changes in these two functions. Can you give me a plan? Thank you.

[ChuckBell]

Sorry for the delay. I will post some suggestions later today.

On Mon, Feb 27, 2023 at 10:10 zytao-01 @.***> wrote:

Sorry, I still don't know how to make changes in these two functions. Can you give me a plan? Thank you.

— Reply to this email directly, view it on GitHub https://github.com/ChuckBell/MySQL_Connector_Arduino/issues/204#issuecomment-1446506118, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6SHYE3OE2KAQWOCGYHWRDWZS7XDANCNFSM6AAAAAAVBLEYMI . You are receiving this because you commented.Message ID: @.***>

[ChuckBell]

Hi. To debug this issue, please examine the code in MySQL_Packet.cpp focusing on the read_packet() method.

First: You will see in line#281-287 that the code reads the packet length. The first thing is to verify this is reading the correct value. It should be about the size of the data (e.g. 1024). If this isn't so, this needs to be fixed.

Second: With the correct packet size, we next must verify all bytes are read. In the same method, look at lines #301-310 to ensure the entire packet is read. If it fails or does not read all the data, this is also a problem.

Finally: Check the code in lines #289-295. You may need to uncomment this code if the data is not being read completely. This introduces a delay that we once needed for older Ethernet cards.

This is where I would concentrate my debugging. I apologize for not having the time to do it myself, but this is the likely location for a problem reading larger rows. It could still be something ancillary to this code, but I'd start here. Hope this helps.

[zytao-01]

Hello, when I look at the code, I know roughly where to modify, but I don't know how to modify the specific. I don't understand what the datagram read by the client is, so I can't modify the code. If you have time, can you provide the specific code? I'm sorry to have taken up too much of your time.

[ChuckBell]

Hi. I tested a row with 1090 bytes. It read fine without any errors. So, the connector code can read a 'wide' row - up to 2047 bytes. I wonder... how many columns are in the row? If you have more than 32, that could be the problem. Otherwise, can you post a row of the data that you are trying to read along with the schema?