HEX uploader, on CrossRoads programmer: verification error when programming Mega2560 with bootloader

[ghost]

This is a continuation of Arduino forum PMs, so i will summarize. I create the HEX file, with the bootloader, using the Arduino IDE. (Menu: Sketch -> Export compiled binary), which creates two HEX files (one with, one without the bootloader). I then load the HEX with the bootloader onto the SD card, and rename to the correct naming convention (like: CODE01.HEX). Upon programming, the HEX is written, but it fails to verify, with code "bd" on the LED display, and also flashes 8 red. the bootloader seems to work, as it can be USB programmed afterward, it just fails verification. By contrast, if i follow the same procedure for the HEX file without the bootloader, it passes verification, and as expected, it no longer is programmable via USB.

In PM you mentioned the order of the sketch and bootloader in the HEX file. when i examined the HEX files directly, the bootloader is added after the sketch. I haven't done the chip detector sketch, but i tried the Blink example as an indicator. pin 13 did not blink with either the HEX with or without the bootloader. though the same sketch programmed via USB worked fine. I will see if i can do the chip detector either tomorrow, or this weekend. very odd results with the testing i have done so far.

[nickgammon]

I meant to use the chip detector sketch separately to try to verify if the bootloader, and the sketch, uploaded or not. Possibly the issue is an incorrect report of a bad verification. I suspect that the change of "page" might be the issue. The page (high-order byte of the address) is different for the bootloader, compared to the sketch. However I eventually got it to work without changing the way the page is handled.

[ghost]

i used the Blink example as it would report if it uploaded by blinking the LED. I will do a more extensive test using the chip detector after work. I have to dig out an UNO to run it on. Something that may be of note, i viewed the HEX files in a hex editor to view the actual file structure better, and i noticed on the file without the bootloader, the line endings are "0D 0A". but in the HEX file with the bootloader, "0A". i don't remember my ASCII, but i bet one is carriage return plus line break, while the other (file with bootloader) is just the line break. If i am reading your code correctly, i think it ignores chars that are not a hex digit, or a colon, but what about the verification?

[ghost]

chip_detect.txt hex_file.txt hex_file.txt is the contents of the HEX file with the bootloader merged in. chip_detect.txt if the results of the chip detector sketch. The hex file is the one i have been trying (my server temp/fan control project), and when i re-uploaded it via the RWP hex programmer hardware. my initial examination is that the program and the bootloader seem to be uploaded correctly, far as i checked. beyond that, i am not sure.

hex_file2.txt

hex_file2.txt is actually the chip detector results after programming the MEGA2560 using the same code, from the Arduino IDE, via USB. there are a lot of 0xFF down in the clip of the program HEX of the RWP programmer, vs the Arduino IDE USB programmer.

~Travis

[ghost]

did i lose you, or just busy?

~Travis

[nickgammon]

If i am reading your code correctly, i think it ignores chars that are not a hex digit, or a colon, but what about the verification?

I don't think that will make any difference. Reading by lines will take either line ending.

chip_detect.txt

Judging by the 0xFF the sketch was not written (only the bootloader).

hex_file2.txt

That looks better at address 0.

Here's what I got. First attempt (using hex_file.txt):

--------- Starting ---------

Attempting to enter ICSP programming mode ...
Entered programming mode OK.
Signature = 0x1E 0x98 0x01 
Processor = ATmega2560
Flash memory size = 262144 bytes.
LFuse = 0xFF 
HFuse = 0xD8 
EFuse = 0xFD 
Lock byte = 0xFF 
Clock calibration = 0x5D 
Actions:
 [E] erase flash
 [F] modify fuses
 [L] list directory
 [R] read from flash (save to disk)
 [V] verify flash (compare to disk)
 [W] write to flash (read from disk)
Enter action:
Programming mode off.
W
Attempting to enter ICSP programming mode ...
Entered programming mode OK.

Choose disk file [ MEGA25~1.HEX ] ...
HEX_FILE.HEX
Processing file: HEX_FILE.HEX
Checking file ...

################################################################
############
Lowest address  = 0x0
Highest address = 0x3FD1D
Bytes to write  = 48336
No bootloader.
Suggest making high fuse = 0xD9 
Attempting to enter ICSP programming mode ...
Entered programming mode OK.
Processing file: HEX_FILE.HEX
Erasing chip ...
Writing flash ...

################################################################
################################################################
##############################################################
Written.
Processing file: HEX_FILE.HEX
Verifying flash ...
Verification error at address 0. Got: 0xFF  Expected: 0xDB 
Verification error at address 1. Got: 0xFF  Expected: 0xC2 
Verification error at address 2. Got: 0xFF  Expected: 0x00 
Verification error at address 3. Got: 0xFF  Expected: 0x00 
Verification error at address 4. Got: 0xFF  Expected: 0x0C 
Verification error at address 5. Got: 0xFF  Expected: 0x94 
Verification error at address 6. Got: 0xFF  Expected: 0xA2 
Verification error at address 7. Got: 0xFF  Expected: 0x09 
Verification error at address 8. Got: 0xFF  Expected: 0x0C 
Verification error at address 9. Got: 0xFF  Expected: 0x94 
Verification error at address A. Got: 0xFF  Expected: 0x77 
Verification error at address B. Got: 0xFF  Expected: 0x09 
Verification error at address C. Got: 0xFF  Expected: 0x0C 
Verification error at address D. Got: 0xFF  Expected: 0x94 
Verification error at address E. Got: 0xFF  Expected: 0x4C 
Verification error at address F. Got: 0xFF  Expected: 0x09 
Verification error at address 10. Got: 0xFF  Expected: 0x0C 

---

Now without changing anything (maybe moving the wires around slightly) the second time it worked:

--------- Starting ---------

Attempting to enter ICSP programming mode ...
Entered programming mode OK.
Signature = 0x1E 0x98 0x01 
Processor = ATmega2560
Flash memory size = 262144 bytes.
LFuse = 0xFF 
HFuse = 0xD9 
EFuse = 0xFD 
Lock byte = 0xFF 
Clock calibration = 0x5D 
Actions:
 [E] erase flash
 [F] modify fuses
 [L] list directory
 [R] read from flash (save to disk)
 [V] verify flash (compare to disk)
 [W] write to flash (read from disk)
Enter action:
Programming mode off.
W
Attempting to enter ICSP programming mode ...
Entered programming mode OK.

Choose disk file [ HEX_FILE.HEX ] ...

Processing file: HEX_FILE.HEX
Checking file ...

################################################################
###########
Lowest address  = 0x0
Highest address = 0x3FD1D
Bytes to write  = 48336
No bootloader.
Suggest making high fuse = 0xD9 
Attempting to enter ICSP programming mode ...
Entered programming mode OK.
Processing file: HEX_FILE.HEX
Erasing chip ...
Writing flash ...

################################################################
################################################################
##############################################################
Written.
Processing file: HEX_FILE.HEX
Verifying flash ...

################################################################
###############################################################
#############################################################
No errors found.
No bootloader.
Setting high fuse = 0xD9 
Done.

---

I'm not quite sure what that means. :)

[ghost]

Now without changing anything (maybe moving the wires around slightly) the second time it worked:

my first thought is that i could have a loose connection, though being Robert's board, i doubt that. it verifies fine without the bootloader, so the connections should be fine.

as just a test, i reloaded your chip detector sketch onto an UNO, using my RWP programmer, with the bootloader merged into the HEX file, and it verified good. so it seems to be isolated to the Mega2560. i kind of guessed that, but i wanted to be sure.

interestingly, if i upload the HEX file that comes with the Arduino IDE for burning the bootloader (C:\Users\Travis\Documents\Backup\Arduino_1-8-5\Arduino\hardware\arduino\avr\bootloaders\stk500v2\Mega2560-prod-firmware-2011-06-29.hex), which is basically a blink sketch with the bootloader merged in, it programs and verifies just fine.

~Travis

[nickgammon]

I am starting to suspect that the "page change" instruction is not being correctly issued. This is the line in the middle of the file:

020000023000CC

This is opcode 0x02 which changes the segment address to 0x3000. (The first 0x02 is the length of the data - two bytes, the last 0xCC is the sumcheck).

---

Try adding this line (underneath) to the very start of the hex file (ie. make it the first line):

:020000020000FC

That resets the page to 0x0000. Possibly the Atmega2560 defaults to some undefined page when it resets.

[ghost]

I haven't done a chip detect on the resulting program yet, but it failed the verification. next chance i get, i plan on making a dedicated chip detector with a stand-alone 328P, should make it easier. ~Travis

[nickgammon]

I thought the problem might have been the extended address byte not being reset to zero, but the code in getSignature does in fact reset it to zero:

      // make sure extended address is zero to match lastAddressMSB variable
      program (loadExtendedAddressByte, 0, 0); 

[ghost]

something i noticed in the HEX file generated by the Arduino IDE. the file with no bootloader looks like it has 2 groups, and the one with the bootloader has 3. not sure what the middle group is, but it looks to me like the bootloader is at the end of the file, rather than the beginning. does this have any significance? i will try moving it around after work and see if it makes a difference. (on lunch break)

~Travis

[nickgammon]

Try reviewing the meaning of the lines. From the source:

/*
Line format:

  :nnaaaatt(data)ss

  Where:
  :      = a colon

  (All of below in hex format)

  nn     = length of data part
  aaaa   = address (eg. where to write data)
  tt     = transaction type
           00 = data
           01 = end of file
           02 = extended segment address (changes high-order byte of the address)
           03 = start segment address *
           04 = linear address *
           05 = start linear address *
  (data) = variable length data
  ss     = sumcheck

            * We don't use these

*/

Check out the "tt" parts of the lines (the transaction type). Looking at that it is pretty clear what each line does.

[ghost]

I am starting to think the verification problem is not with the bootloader, but the sketch itself. I haven't done a chip detect to confirm, but i used the Blink example as a sort of standardized test. I created 4 HEX files: UNO - no boot, UNO - boot, 2560 - no boot, 2560 - boot. Both UNO files, on an UNO of course, resulted with pin13 LED blinking. Both 2560 files, on a MEGA2560, resulted in no blinking of pin13 LED. in both examples, the HEX files with the bootloader could be USB programmed, while those without could not, as expected. so while we were looking at the bootloader for errors, the problem is in the sketch portion.

~Travis

[nickgammon]

Yes, the time I could reproduce your problem it was the sketch that wasn't written, not the bootloader. This made me think that perhaps the wrong page (extended segment address) was being programmed.

Try re-ordering the file so that the bootloader is first in it. Make sure that the TT 0x02 line (which changes the extended address) is in the right place before the bootloader and also before the sketch.

I gave earlier in this thread what the line should be to change back to address 0x0000.

:020000020000FC

[ghost]

Interesting results. I will attach the HEX file i altered. The unaltered hex from Arduino IDE, for 2560, with bootloader: Blink.ino.with_bootloader.mega.txt

Here is the file with the TT 0x02 line changed, but bootloader position left alone: CODE00.txt

now, interestingly, it passed verification. but i don't think the program loaded right, as the pin13 LED did not blink. (i really got to get that chip detector rigged back up) to satisfy my scientific curiosity, i loaded the non-bootloader HEX from the blink example, and loaded it onto the Mega2560, and it passed verify, but did not blink. i am thinking there may be something wrong (or different) with how the Arduino IDE exports the HEX files for a 2560.

~Travis

[nickgammon]

No, you need to have two TT 0x02 lines. ie.

:020000020000FC   -- address 0x00xxxx
( blink sketch )
:020000023000CC    -- address 0x30xxxx
( bootloader )
:00000001FF    -- end of file

[ghost]

CODE00.txt CODE01.txt 00 - with bootloader, 01 - without. 00 fails verify, 01 passes, neither blink. I think it would be helpful to be able to debug and alter how the Arduino IDE creates the HEX files. seems like it is missing something in the sketch portion. if i was to create the HEX files with a manual compile, perhaps i could see what is going on. I have never done that though. i use CodeBlocks (i like the code highlighting, and it even highlights the correct columns for HEX files) to write the code, and then open the file in Arduino to either compile and export to HEX, or directly upload.

~Travis

[nickgammon]

I think you need to set up another Arduino with my chip detector sketch on it. If it passes the verification, but doesn't blink, either the verification is wrong, or the blink sketch isn't blinking for some other reason. For the sketch that verifies, is the bootloader operative?

[ghost]

I will setup a chip detect when i get home from work. and yes, the bootloader is operative, at least in previous tests. I will confirm after work.

~Travis

[ghost]

without bootloader: no_boot.txt

With bootloader: boot.txt

these are from the same files last posted. it looks fairly plain to see why the program doesn't result in a Blink. all 0xFF.

( sorry for the day later posting, i had a migraine after work yesterday) ~Travis

[ghost]

For the sketch that verifies, is the bootloader operative?

sorry, i misunderstood, the answer is no.

~Travis

[nickgammon]

Can you please try my "hex uploader" sketch. This requires you to interface with the code via the serial monitor. You just have to connect up an ICSP socket as documented on the web page: http://www.gammon.com.au/uploader

I have been using that because I wanted to see the debugging messages it outputs. That is more helpful than trying to decode blinking lights.

[ghost]

i used my CrossRoads hardware, as it has an FTDI port, conveniantly, and just reprogrammed it, temporarily. upload001.txt

~Travis

[ghost]

as a hunch, i recompiled the Blink example to create fresh HEX files. the thinking was, that the programmer sketch provided by RWP was version 1.25i, and on your GitHub, version 1.37. so, unknown to me, we were working with vastly different versions. So, long story longer, i have success! programmed the Blink sketch, with the bootloader, it verified, and the MEGA2560 is blinking! I will go ahead and change the variables to make it compatible with the hardware, and report back in a moment.

~Travis

[ghost]

Hmm, seems there are no longer any variables to support the standalone HEX programmer hardware. Was it phased out?

(edit) i just realized, i am not sure if i said or not, i have RWP's dual display, 256 hex file standalone programmer.

~Travis

[ghost]

Ok, i found the latest source for the standalone programmer, and made the variable changes, and uploaded it to my RWP programmer. i tested with a fresh HEX file compile, and the upload (with bootloader) passed verification, but i am unable to upload the Blink sketch via USB, so the bootloader is non-functional. i will re-install the HEX uploader (via serial user interface), and see if it too produced a bad bootloader. back in a minute.

[ghost]

upload002.txt notice the line "No bootloader." so the bootloader is not being read from the HEX, maybe? or it is stored in the wrong address.

~Travis

[nickgammon]

As I mentioned earlier somewhere, if the code finds the sketch starts at 0x0000 then it assumes no bootloader and sets the fuses accordingly. This is just a fuse issue.

To hard-code the correct fuses for this particular processor (Atmega2560) to boot into the bootloader replace the contents of function updateFuses with:

bool updateFuses (const bool writeIt)
  {
  if (writeIt)
    writeFuse (0xD8, writeHighFuseByte);  // boot into bootloader, size is 8192 bytes

  return false;
  }

Note to others reading this: This change is only for this processor (Atmega2560) where you are uploading a file with both code (starting at 0x0000) and a bootloader (starting at 0x3E000) in the one file. For other processors I suggest looking at http://www.engbedded.com/fusecalc and working out an appropriate value for the fuse.

I can't automatically deduce this, because you might simply have a large sketch (which starts at 0x0000) and which extends into the bootloader memory. In this case there would be no bootloader.

If you get the fuse value wrong you can "brick" the processor because you might turn off ICSP programming, which means you won't be able to enter programming mode, unless you have a high-voltage programmer, so please take care.

[ghost]

I think rather than accidentally bricking a non-2560, i will leave the fuses be. i guess if i need to bootload program a processor, i can burn a bootloader, then program. the processor i wanted the standalone programmer for will be in a location i can't use a laptop easily anyway (hence the standalone programmer). IOW, i'm giving up for now. I thank you profusely for your time and effort, as it now verifies, where before it did not.

~Travis

[nickgammon]

I think rather than accidentally bricking a non-2560, i will leave the fuses be

You could conceivably throw in an "if" test, leaving in the existing code, but testing if a certain processor, do it differently.

For example:

// returns true if error, false if OK
bool updateFuses (const bool writeIt)
  {
  if (strcmp (currentSignature.desc, "ATmega2560" == 0))
    {
    if (writeIt)
      writeFuse (0xD8, writeHighFuseByte);  // boot into bootloader, size is 8192 bytes

    return false;
    }  // end of ATmega2560

   // --- existing updateFuses function code here ...

}  // end of updateFuses