Wireless bootloading fails on large files

[nseidle]

I've been working with Impeeduino to wirelessly bootload my Arduino projects. I've found a bug that I can't quite squash. The device is loading code flawlessly, but the agent is parsing the HEX file incorrectly. Here's a comparison of the HEX file on the right that should be sent, and what is read back from the board on the left using an external programmer:

You can see the agent is compressing the codewords together. This causes major havoc on the Arduino. I know the bug is in function program(hex) and I'm trying to wrap my head around squirrel. But if you guys have a chance I would love some guidance. Thanks!

[electricimp]

Looking at the file, it appears to be identical (based on a short inspection), it’s just the hex file you’re programming has a short line in it.

On the right, the first yellow highlighted line contains only 8 bytes (starts with :08), at address 0x2A40. The record type is 00 then there 8 bytes and the checksum. The next line is address 0x2A48. If you look at the first 8 bytes of data at address 0x2A48 (i.e. on the next line) - 242C77696E646469 - you’ll notice that’s the second half of the 16 byte line on the left.

The intel hex format - see http://en.wikipedia.org/wiki/Intel_HEX - has address and length per line. My guess is your compiler generating the hex file has got to the end of a section and hence has output a short hex line before resuming (without skipping any bytes).

Are you actually seeing an issue here, or are you just worried about the verify?

On Sunday, March 2, 2014 at 13:49 , Nathan Seidle wrote:

I've been working with Impeeduino (https://github.com/electricimp/reference/tree/master/hardware/impeeduino) to wirelessly bootload my Arduino projects. I've found a bug that I can't quite squash. The device is loading code flawlessly, but the agent is parsing the HEX file incorrectly. Here's a comparison of the HEX file on the right that should be sent, and what is read back from the board on the left using an external programmer:

You can see the agent is compressing the codewords together. This causes major havoc on the Arduino. I know the bug is in function program(hex) and I'm trying to wrap my head around squirrel. But if you guys have a chance I would love some guidance. Thanks!

— Reply to this email directly or view it on GitHub (https://github.com/electricimp/reference/issues/8).

[nseidle]

Hmm. I disagree. Perhaps my image is not clear.

On the right, the correct values at 0x2A48 are 0x242C77.... On the left, the incorrect values at 0x2A48...

AH! I see what you're saying! I agree with you now. I'll keep digging. There is something causing issues.

[nseidle]

Sorry for the original confusion. I still believe there is a bug.

On the left is the HEX dump from a board after programming it in Arduino. On the right is that same HEX file (but not the dump) sent over the Imp. This is happening suspiciously around the 0x2A40 break shown in the original HEX.

Here are the three files : Original Arduino output, post programming output, and post Imp programming output.

[ginomiglio]

If you do this multiple times, do you get the same bad bytes? That would point to a parsing error; otherwise, it could be a programming error (i.e. during the flash program using the arduino bootloader, for whatever reason).

What’s the hardware connection here? Is the FTDI still connected to the arduino?

Hugo Fiennes electric imp, inc

On Sunday, March 2, 2014 at 15:49 , Nathan Seidle wrote:

Sorry for the original confusion. I still believe there is a bug.

On the left is the HEX dump from a board after programming it in Arduino. On the right is that same HEX file (but not the dump) sent over the Imp. This is happening suspiciously around the 0x2A40 break shown in the original HEX. Here are the three files (http://ge.tt/5gsfXXN1) : Original Arduino output, post programming output, and post Imp programming output.

— Reply to this email directly or view it on GitHub (https://github.com/electricimp/reference/issues/8#issuecomment-36472272).

[nseidle]

I am deep into modifying the parser so I'll try to back out, but I am confident it's reproducible and a problem with the parser (not a serial byte corruption).

Hardware connection: This tutorial. Power via USB onto an Arduino with SparkFun Imp shield re-routed to Arduino pins 0/1. The FTDI is behind two 1k resistors on Arduino Unos so the Imp overrides the serial. I've also verified serial comm with a logic analyzer. The device is working correctly. I believe the wrong bytes are being sent by the agent.

I think there's an issue with the 128 blob. data.tell() doesn't check to see if it is over the 128 byte limit so I think there's an array that is wrapping and/or getting corrupted.

[nseidle]

Doing multiple wireless programming cycles the same incorrect bytes are written over and over.

[blindman2k]

It's been a while since I have touched this code (many other projects have pushed it out of my brain) but I have another HEX project coming up so am happy to help ensure any bugs are resolved. Keep me up to date with how you are progressing and I will give it a check when I can.

Aron.

[blindman2k]

The issue doesn't appear to be related to the 0x2A40 break. It's four lines later. Both blocks that are in error are repeated later in the file too. I think the blob passing 128 bytes is the culprit tho. I will find the little bugger.

[blindman2k]

Try changing:

if (tell == data.len()) {

to:

if (tell => 128) {

[blindman2k]

Ok, I have fixed it. Check the latest version in git. Basically, every packet sent (except the last) MUST be 128 bytes long. This means I have to take half packets sometimes.

[blindman2k]

... and now it is rewritten to build the entire binary in memory and then chunk and send it to the device. Much much simpler and can tolerate any type of weird Hex files.