Unable to burn bootloader with Atmel-ICE in Arduino IDE 1.8.13

[kristofmulier]

I'm trying to burn an Arduino bootloader on an Arduino UNO board with the Atmel-ICE probe. Unfortunately, avrdude throws an error message when verifying the bytes.

1. Hardware setup

My hardware setup looks like this:

By the way - I'm working on a 64-bit Windows 10 PC.

2. Atmel-ICE driver

At first I thought that the Atmel-ICE doesn't require installation, because Windows recognizes it as an HID-device. However, I quickly learned that avrdude - launched by the Arduino IDE - needs another driver to interact with the Atmel-ICE. Therefore, I installed the libusb-win32 driver with Zadig (just as suggested on the GitHub thread https://github.com/arduino/Arduino/issues/4368):

The driver installation succeeded, and I observe a change in the Windows Device Manager. The Atmel-ICE probe no longer appears as an HID-device but as a libusb-win32 device instead:

I believe the Atmel-ICE is now ready to be used with avrdude.

3. Software setup

I'm running a fresh-installed Arduino IDE (version 1.8.13) on a Windows 10 PC. First I activate maximal verbosity for the console output (File > Preferences > Show verbose output). Next, I select the probe: Tools > Programmer > Atmel-ICE (AVR)

Next I select my board: Tools > Board > Arduino AVR Boards > Arduino UNO

Finally, I burn the bootloader: Tools > Burn Bootloader

In the console output, I can see two avrdude commands being launched.

3.1 First avrdude command

The first avrdude command looks like this:

avrdude -CC:\Program Files (x86)\.../avrdude.conf
        -v
        -patmega328p
        -catmelice_isp
        -Pusb
        -e
        -Ulock:w:0x3F:m
        -Uefuse:w:0xFD:m
        -Uhfuse:w:0xDE:m
        -Ulfuse:w:0xFF:m

the output for this command is:

avrdude: Version 6.3-20190619
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Program Files (x86)\Arduino\hardware\tools\avr/etc/avrdude.conf"

         Using Port                    : usb
         Using Programmer              : atmelice_isp
avrdude: usbhid_open(): No device found
avrdude: usbdev_open(): Found Atmel-ICE CMSIS-DAP, serno: J42700007942
avrdude: Found CMSIS-DAP compliant device, using EDBG protocol
         AVR Part                      : ATmega328P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : JTAG3_ISP
         Description     : Atmel-ICE (ARM/AVR) in ISP mode
         Vtarget         : 5.0 V
         SCK period      : 125.00 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: erasing chip
avrdude: reading input file "0x3F"
avrdude: writing lock (1 bytes):

Writing | ################################################## | 100% 0.00s

avrdude: 1 bytes of lock written
avrdude: verifying lock memory against 0x3F:
avrdude: load data lock data from input file 0x3F:
avrdude: input file 0x3F contains 1 bytes
avrdude: reading on-chip lock data:

Reading | ################################################## | 100% 0.01s

avrdude: verifying ...
avrdude: WARNING: invalid value for unused bits in fuse "lock", should be set to 1 according to datasheet
This behaviour is deprecated and will result in an error in future version
You probably want to use 0xff instead of 0x3f (double check with your datasheet first).
avrdude: 1 bytes of lock verified
avrdude: reading input file "0xFD"
avrdude: writing efuse (1 bytes):

Writing | ################################################## | 100% 0.09s

avrdude: 1 bytes of efuse written
avrdude: verifying efuse memory against 0xFD:
avrdude: load data efuse data from input file 0xFD:
avrdude: input file 0xFD contains 1 bytes
avrdude: reading on-chip efuse data:

Reading | ################################################## | 100% 0.00s

avrdude: verifying ...
avrdude: 1 bytes of efuse verified
avrdude: reading input file "0xDE"
avrdude: writing hfuse (1 bytes):

Writing | ################################################## | 100% 0.09s

avrdude: 1 bytes of hfuse written
avrdude: verifying hfuse memory against 0xDE:
avrdude: load data hfuse data from input file 0xDE:
avrdude: input file 0xDE contains 1 bytes
avrdude: reading on-chip hfuse data:

Reading | ################################################## | 100% 0.00s

avrdude: verifying ...
avrdude: 1 bytes of hfuse verified
avrdude: reading input file "0xFF"
avrdude: writing lfuse (1 bytes):

Writing | ################################################## | 100% 0.09s

avrdude: 1 bytes of lfuse written
avrdude: verifying lfuse memory against 0xFF:
avrdude: load data lfuse data from input file 0xFF:
avrdude: input file 0xFF contains 1 bytes
avrdude: reading on-chip lfuse data:

Reading | ################################################## | 100% 0.00s

avrdude: verifying ...
avrdude: 1 bytes of lfuse verified

avrdude done.  Thank you.

That looks good.

3.2 Second avrdude command

The second avrdude command is:

avrdude -CC:\Program Files (x86)\.../avrdude.conf
        -v
        -patmega328p
        -catmelice_isp
        -Pusb
        -Uflash:w:C:\Program Files (x86)\.../optiboot_atmega328.hex:i
        -Ulock:w:0x0F:m 

with the following output:

avrdude: Version 6.3-20190619
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "C:\Program Files (x86)\Arduino\hardware\tools\avr/etc/avrdude.conf"

         Using Port                    : usb
         Using Programmer              : atmelice_isp
avrdude: usbhid_open(): No device found
avrdude: usbdev_open(): Found Atmel-ICE CMSIS-DAP, serno: J42700007942
avrdude: Found CMSIS-DAP compliant device, using EDBG protocol
         AVR Part                      : ATmega328P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : JTAG3_ISP
         Description     : Atmel-ICE (ARM/AVR) in ISP mode
         Vtarget         : 5.0 V
         SCK period      : 125.00 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: NOTE: "flash" memory has been specified, an erase cycle will be performed
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: reading input file "C:\Program Files (x86)\Arduino\hardware\arduino\avr/bootloaders/optiboot/optiboot_atmega328.hex"
avrdude: writing flash (32768 bytes):

Writing | ################################################## | 100% -0.00s

avrdude: 32768 bytes of flash written
avrdude: verifying flash memory against C:\Program Files (x86)\Arduino\hardware\arduino\avr/bootloaders/optiboot/optiboot_atmega328.hex:
avrdude: load data flash data from input file C:\Program Files (x86)\Arduino\hardware\arduino\avr/bootloaders/optiboot/optiboot_atmega328.hex:
avrdude: input file C:\Program Files (x86)\Arduino\hardware\arduino\avr/bootloaders/optiboot/optiboot_atmega328.hex contains 32768 bytes
avrdude: reading on-chip flash data:

Reading | ################################################## | 100% -0.00s

avrdude: verifying ...
avrdude: 32768 bytes of flash verified
avrdude: reading input file "0x0F"
avrdude: writing lock (1 bytes):

Error while burning bootloader.
Writing | ################################################## | 100% 0.00s

avrdude: 1 bytes of lock written
avrdude: verifying lock memory against 0x0F:
avrdude: load data lock data from input file 0x0F:
avrdude: input file 0x0F contains 1 bytes
avrdude: reading on-chip lock data:

Reading | ################################################## | 100% 0.01s

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0xff != 0x0f
avrdude: verification error; content mismatch

avrdude done.  Thank you.

3.3 The error message

As you can see, avrdude throws this error message:

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0xff != 0x0f
avrdude: verification error; content mismatch

What should I do to solve this?

[matthijskooijman]

What seems to be happening is that the second command tries to set the lock bits so that the bootloader or sketch cannot overwrite bootloader, but for some reason, that fails.

There have been some issues with verification of fuses and lock bytes wrt the bits that are unused (some programmers read them back as 0, some as 1), but the avrdude version that you are using has been patched by Arduino to ignore these unused bytes entirely when verifiying. In this case, I think that the upper two bits are unused and the lower six (0x3F) are used. The second avrdude programs a value of 0x0F, so that's lock bit 4 and 5 set (0 is set, 1 is cleared), but the value read back is 0xFF, which has lock bit 4 and 5 cleared (0). Also, bit 6 and 7 are different, but those are unused bits, so that's ok.

I'm not sure why setting the lock bits fails, though...

[kristofmulier]

Hi @matthijskooijman , Thank you very much for your insights! I'm new to the Arduino ecosystem and AVR chips, so I'm not completely following your explanations. However - after reading your reply - it looks like there must be a workaround. Do you have a suggestion, things I could try? I don't mind playing around in the cmd console ;-)

I believe there is no Hardware issue. The same error message appears when I try to burn the bootloader into my other Arduino UNO SMD board (I've got an smd and a dip version).

PS: By the way - the reason I'm diving into Arduino has everything to do with this new free IDE for microcontrollers: Embeetle IDE (see https://embeetle.com). I'm currently in the process of adding Arduino support there.

[matthijskooijman]

Thank you very much for your insights! I'm new to the Arduino ecosystem and AVR chips, so I'm not completely following your explanations.

If you have specific questions, just ask.

However - after reading your reply - it looks like there must be a workaround. Do you have a suggestion, things I could try? I don't mind playing around in the cmd console ;-)

Well, it looks like the bootloader was flashed succesfully, it's just the lock bits that could not be set for some reason. So I'd expect that your board works, it's just not as well-protected as it should.

You could also try running avrdude manually to read the lock byte back, to see if it the write really failed, or it's just the verify that fails somehow (maybe the lock bytes only take effect after a reset or so?). For that, I think you can use this command (haven't fully tested it though):

avrdude -CC:\Program Files (x86)\.../avrdude.conf
        -v
        -patmega328p
        -catmelice_isp
        -Pusb
        -Ulock:r:-:h

I'm not 100% sure that using - for "write to stdout" works, if not you might end up with a file called -. Also, replace the avrdude.conf argument with the full path.

PS: By the way - the reason I'm diving into Arduino has everything to do with this new free IDE for microcontrollers: Embeetle IDE (see https://embeetle.com). I'm currently in the process of adding Arduino support there.

If you haven't already, check out arduino-cli, that's a convenient CLI tool that can handle the entire Arduino build process and might be useful to use as a backend for other IDEs (it's not yet used directly by the Java IDE yet, that still uses arduino-builder, but that again includes parts of arduino-cli).

[kristofmulier]

Hi @matthijskooijman , Thank you very much for your reply!

I just tried something. I entered the exact same commands that the Arduino IDE prints in its console into a Windows terminal.

1. Same commands - but in Windows Terminal

I opened a Windows Terminal and entered the two avrdude commands just as I see them in the Arduino IDE console.

1.1 First avrdude command

The first command I entered is this one:

"C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avrdude" -C"C:\Program Files (x86)\Arduino\hardware\tools\avr/etc/avrdude.conf" -v -patmega328p -catmelice_isp -Pusb -e -Ulock:w:0x3F:m -Uefuse:w:0xFD:m -Uhfuse:w:0xDE:m -Ulfuse:w:0xFF:m

Restyled and reformatted, it looks like this (see also my original post):

avrdude -C C:/Program Files (x86)/.../avrdude.conf
        -v
        -patmega328p
        -catmelice_isp
        -Pusb
        -e
        -Ulock:w:0x3F:m
        -Uefuse:w:0xFD:m
        -Uhfuse:w:0xDE:m
        -Ulfuse:w:0xFF:m

The output is exactly the same as before (see my original post), with a few exceptions. After printing the table with memory info, avrdude prints this line:

avrdude: AVR device initialized and ready to accept instructions

In the original output (from the Arduino IDE console), the following is printed right after that line:

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: erasing chip
avrdude: reading input file "0x3F"

In the Windows terminal output, I see this instead:

Reading | ################################################## | 100% 0.02s

avrdude: Device signature = 0x1e950f (probably m328p)
avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as DE
avrdude: safemode: efuse reads as FD
avrdude: erasing chip
avrdude: reading input file "0x3F"

Another difference is all the way at the end of the output. The original output (from the Arduino IDE console, see original post) ends like this:

Reading | ################################################## | 100% 0.00s

avrdude: verifying ...
avrdude: 1 bytes of lfuse verified

avrdude done.  Thank you.

While the Windows terminal outputs this:

Reading | ################################################## | 100% 0.02s

avrdude: verifying ...
avrdude: 1 bytes of lfuse verified

avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as DE
avrdude: safemode: efuse reads as FD
avrdude: safemode: Fuses OK (E:FD, H:DE, L:FF)

avrdude done.  Thank you.

1.2 Second avrdude command

The second avrdude command I enter is this one:

"C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avrdude" -C"C:\Program Files (x86)\Arduino\hardware\tools\avr/etc/avrdude.conf" -v -patmega328p -catmelice_isp -Pusb -Uflash:w:"C:\Program Files (x86)\Arduino\hardware\arduino\avr/bootloaders/optiboot/optiboot_atmega328.hex":i -Ulock:w:0x0F:m

Restyled and reformatted, it looks like this (see also my original post):

avrdude -C C:/Program Files (x86)/.../avrdude.conf
        -v
        -patmega328p
        -catmelice_isp
        -Pusb
        -Uflash:w:"C:/Program Files (x86)/.../optiboot_atmega328.hex":i
        -Ulock:w:0x0F:m 

The output is the same as before, with the exception that the output in the Arduino IDE ends like this:

Reading | ################################################## | 100% 0.01s

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0xff != 0x0f
avrdude: verification error; content mismatch

avrdude done.  Thank you.

and the output in the Windows terminal ends like this:

Reading | ################################################## | 100% 0.02s

avrdude: verifying ...
avrdude: WARNING: invalid value for unused bits in fuse "lock", should be set to 1 according to datasheet
This behaviour is deprecated and will result in an error in future version
You probably want to use 0xcf instead of 0x0f (double check with your datasheet first).
avrdude: 1 bytes of lock verified

avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as DE
avrdude: safemode: efuse reads as FD
avrdude: safemode: Fuses OK (E:FD, H:DE, L:FF)

avrdude done.  Thank you.

2. Conclusions?

I don't know what to conclude from this. How is it possible that the same avrdude commands give a different results when they run in the Arduino IDE console compared to manually entering them in the Windows terminal?

One thing that comes to my mind is a timing issue. In the Arduino IDE, they run rapidly one after the other. When I enter them manually, there is considerably more time between them. However, that doesn't explain the difference in the output of the first command.

To test the timing-issue-theory, I put both commands in a .bat file. I ran the .bat file, and it worked. As both commands now run from the .bat file, there is almost no time in between them. This should mimic the behaviour of the Arduino IDE console very close.

3. Additional question

Well - it seems like it does work in the Windows console, and that's what ultimately matters to me (of course, it would be nice if it gets fixed in the Arduino IDE as well).

However, the following output scares me a little (it's at the end of the output for the second command):

Reading | ################################################## | 100% 0.02s

avrdude: verifying ...
avrdude: WARNING: invalid value for unused bits in fuse "lock", should be set to 1 according to datasheet
This behaviour is deprecated and will result in an error in future version
You probably want to use 0xcf instead of 0x0f (double check with your datasheet first).
avrdude: 1 bytes of lock verified

avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as DE
avrdude: safemode: efuse reads as FD
avrdude: safemode: Fuses OK (E:FD, H:DE, L:FF)

avrdude done.  Thank you.

Especially this phrase: "This behaviour is deprecated and will result in an error in future version"

I assume it has to do with the last flag in the second command: -Ulock:w:0x0F:m. Should I change that flag into: -Ulock:w:0xCF:m? The output suggests to look into the datasheet. However, I am not familiar with this chip, so I have no idea where to look in the datasheet.

[matthijskooijman]

Hm, interesting. I had also noticed that the warning appears the first time but not the second time, but re-reading the avrdude patch (https://github.com/arduino/avrdude-build-script/blob/master/avrdude-6.3-patches/80-Avoid-failing-fuse-check-if-different-bits-are-reserved.patch) I see that the patch is only shown when verification succeeds (with only differences in the unused bits), not when verification fails. So the main difference is that your manual run succeeds, while the IDE run somehow does not. I'm not sure why your manual run shows more (debug) output wrt "safemode" (extra fuse comparisons after programming).

Especially this phrase: "This behaviour is deprecated and will result in an error in future version"

I assume it has to do with the last flag in the second command: -Ulock:w:0x0F:m. Should I change that flag into: -Ulock:w:0xCF:m? The output suggests to look into the datasheet. However, I am not familiar with this chip, so I have no idea where to look in the datasheet.

This relates to the unused lock bits I mentioned before. Originally, you'd need 0x0F for some programmers and 0xCF for others. With an Arduino-specific patch, the unused bits are ignored, but also this warning was added to encourage users of the avrdude binary to switch to 0xCF instead. In this case, that means changing the lock byte values in the boards.txt file of the AVR core. It seems we already made these changes for fuse bytes (https://github.com/arduino/Arduino/pull/5182) but not for the lock bytes (see https://github.com/arduino/Arduino/pull/6601).

So this is something we should fix in boards.txt, but I don't think this causes your problem.

To test the timing-issue-theory, I put both commands in a .bat file. I ran the .bat file, and it worked. As both commands now run from the .bat file, there is almost no time in between them. This should mimic the behaviour of the Arduino IDE console very close.

Hm, so maybe not a timing problem between both commands, then.

However, what you said reminded me of lock-byte timing issue I had observed with the JTAGICE3 years ago. I did a bit of digging and found my bug report: https://savannah.nongnu.org/bugs/?func=detailitem&item_id=42267 This was fixed in avrdude also years ago, but the "fix" was adding a bit of a delay, so maybe the delay isn't always enough somehow? In the report it was also suggested that adding more -v options could slow down things enough to fix them, so maybe you could try editing platform.txt and add two or three more -v options to the second avrdude command as ran by the IDE, here: https://github.com/arduino/ArduinoCore-avr/blob/60f0d0b125e06dbf57b800192c80e5f60d681438/platform.txt#L121

[kristofmulier]

Hi @matthijskooijman , You make an interesting analysis there 👍 Basically, there are two issues at hand:

1. Unused lock bits

The unused lock bits issue is indicated by the warning:

avrdude: verifying ...
avrdude: WARNING: invalid value for unused bits in fuse "lock", should be set to 1 according to datasheet
This behaviour is deprecated and will result in an error in future version
You probably want to use 0xcf instead of 0x0f (double check with your datasheet first).
avrdude: 1 bytes of lock verified

As you say, this issue is most probably not causing the avrdude crash I experience in the Arduino IDE. It's an unrelated problem. However, it's still worthwile to fix it for our implementation in Embeetle IDE. Do you suggest we use the flag -Ulock:w:0x0F:m or rather -Ulock:w:0xCF:m? I'm referring to the following setup:

• board: Arduino UNO
• flash/debug probe: Atmel-ICE
• action: burn bootloader

The warning suggests to check the datasheet first before deciding. Unfortunately, I have not enough experience with AVR microcontrollers to make an informed decision here. I hope you can give me a suggestion.

2. avrdude crash in Arduino IDE

You're absolutely right about this:

"So the main difference is that your manual run succeeds, while the IDE run somehow does not."

Very intriguing, indeed.

"However, what you said reminded me of lock-byte timing issue I had observed with the JTAGICE3 years ago. [...] In the report it was also suggested that adding more -v options could slow down things enough to fix them, so maybe you could try editing platform.txt and add two or three more -v options to the second avrdude command as ran by the IDE [...]"

Interesting observation!

I just added a -v option at the end of line 121 in platform.txt:

tools.avrdude.bootloader.pattern="{cmd.path}" "-C{config.path}" {bootloader.verbose} -p{build.mcu} -c{protocol} {program.extra_params} "-Uflash:w:{runtime.platform.path}/bootloaders/{bootloader.file}:i" -Ulock:w:{bootloader.lock_bits}:m -v

The error persisted. Even with two -v flags. But adding a third(!) one solved the issue. The bootloader burn procedure seems to work now in Arduino IDE! However, the verbosity is ridiculous. The output is more than 1100 lines and around 650.000 characters. You can download the output file here (I've put it on our server):

https://embeetle.com/downloads/misc/extremely_verbose_output.txt

Shouldn't this be fixed in a proper way in avrdude? Do you have a suggestion how to go on from here?

[matthijskooijman]

Do you suggest we use the flag -Ulock:w:0x0F:m or rather -Ulock:w:0xCF:m?

You should use 0xCF for the 328p, that has the unused bits set to 1.

Note that this does assume using the Arduino-patched version of avrdude, since upstream has not merged this patch yet. Without this patch, 0x0F will fail to verify on some programmers and 0xCF will fail to verify on others.

Shouldn't this be fixed in a proper way in avrdude? Do you have a suggestion how to go on from here?

Hm, you testing suggests that maybe the delay put into avrdude is not enough somehow. IIRC they put in 10ms while (later) Atmel suggested they use 20ms, so maybe the delay should be extended, then.

I'll dig out my JTAGICE3 (which is very similar to the Atmel ICE) and see if I can reproduce this in the coming days. I'll also see if I can build a custom avrdude with a bit more delay, to see if that solves the problem reliably. If so, we can provide a patch to avrdude upstream, see if they want to merge that (and in the meanwhile, probably apply the patch in the Arduino version too).

[kristofmulier]

Hi @matthijskooijman ,

"I'll dig out my JTAGICE3 (which is very similar to the Atmel ICE) and see if I can reproduce this in the coming days."

That's a great idea!

Please let me know if you can't reproduce the issue with the JTAGICE3. In that case, I'd like to send you the funds to buy a new Atmel-ICE for this test. I can send the funds over PayPal or any other way you like.

[kristofmulier]

Hi @matthijskooijman ,

Some more interesting news. I just tried to run the two avrdude commands from the console in Embeetle IDE, and I've got the exact same problem as in Arduino IDE. So there must be a subtle (timing?) difference when entering these commands directly in a Windows terminal vs invoking them from another software. I don't know how Arduino IDE invokes its console commands, but Embeetle does it from Python with a subprocess.

In Embeetle IDE, I added an artificial wait time (up to several seconds) between both avrdude command invocations - but the error persists.

However, just like in Arduino IDE, the bootloader burning works when I add three(!) -v flags to the second command. Unfortunately, that is not a viable solution. It overloads the user with extremely verbose output.

Thanks a lot for your help :-)

[matthijskooijman]

Please let me know if you can't reproduce the issue with the JTAGICE3. In that case, I'd like to send you the funds to buy a new Atmel-ICE for this test. I can send the funds over PayPal or any other way you like.

If I can't, I'll probably try to send you a custom avrdude build to test, probably easier. Also, I have access to an Atmel ICE in the office of a customer, but due to Corona I'm not yet sure when I'll be there again. In any case, thanks for the offer!

So there must be a subtle (timing?) difference when entering these commands directly in a Windows terminal vs invoking them from another software. I don't know how Arduino IDE invokes its console commands, but Embeetle does it from Python with a subprocess.

Thinking on this a bit more, I suspect that the output handling might be difference here. Probably outputting to a console is slightly slower than to a pipe (as used by the IDE), leading to this difference in timing. You could try running in the console with output redirected to a file, I'd guess that would break it.

[kristofmulier]

Hi @matthijskooijman ,

I have access to an Atmel ICE in the office of a customer, but due to Corona I'm not yet sure when I'll be there again.

The offer for a new Atmel-ICE still stands. It's the least I can do for your generous help! I could purchase it on Farnell and send it to your address directly - so you'd have it in one day.

Thinking on this a bit more, I suspect that the output handling might be difference here. Probably outputting to a console is slightly slower than to a pipe (as used by the IDE), leading to this difference in timing.

That could very well be true!

You could try running in the console with output redirected to a file.

Unfortunately, that's not possible in Embeetle IDE. I mean - it would be possible, but it would require too much code refactoring.

PS: please drop me a mail so we can stay in contact: kristof@embeetle.com Hope to hear from you soon!

[matthijskooijman]

Unfortunately, that's not possible in Embeetle IDE. I mean - it would be possible, but it would require too much code refactoring.

I meant in the windows terminal, where until now your manual avrdude call did not break.

PS: please drop me a mail so we can stay in contact:

I'd rather keep this on github as much as possible, for others to read, contribute and for public history. If you do need to contact me privately, my e-mail is in my github profile.

[kristofmulier]

Hi @matthijskooijman , I just tried your suggestion (to redirect the output to a file) in the Windows terminal:

"C:\Program Files (x86)\Arduino\hardware\tools\avr/bin/avrdude" -C"C:\Program Files (x86)\Arduino\hardware\tools\avr/etc/avrdude.conf" -v -patmega328p -catmelice_isp -Pusb -Uflash:w:"C:\Program Files (x86)\Arduino\hardware\arduino\avr/bootloaders/optiboot/optiboot_atmega328.hex":i -Ulock:w:0x0F:m > file.txt 2> errfile.txt

All output is now redirected to files file.txt and errfile.txt. It's a bit weird, but file.txt is totally empty. Seems like avrdude likes to write everything to its stderr channel. Anyway, this is how errfile.txt ends:

Reading | ################################################## | 100% 0.02s

avrdude: verifying ...
avrdude: WARNING: invalid value for unused bits in fuse "lock", should be set to 1 according to datasheet
This behaviour is deprecated and will result in an error in future version
You probably want to use 0xcf instead of 0x0f (double check with your datasheet first).
avrdude: 1 bytes of lock verified

avrdude: safemode: lfuse reads as FF
avrdude: safemode: hfuse reads as DE
avrdude: safemode: efuse reads as FD
avrdude: safemode: Fuses OK (E:FD, H:DE, L:FF)

avrdude done.  Thank you.

That looks fine - no errors.

PS: I dropped you a mail, just to open a communication channel (and to see if I don't end up in your SPAM folder).

[matthijskooijman]

Seems like avrdude likes to write everything to its stderr channel

Yeah, I think it does that. Annoying, because all output is marked red by the IDE, but that's how it is...

That looks fine - no errors.

Ok, so maybe something slightly different that affects timing somehow.

In any case:

• I can reproduce the problem locally with my JTAGICE3
• I discovered that the upstream fix/workaround was made to their SVN trunk, but was never released (the last 6.3 release is from 2016...)
• Then I saw that the Arduino avrdude uses SVN trunk as a basis for their builds (though lagging behind), but this particular commit was reverted in https://github.com/facchinm/avrdude/commit/6b0863177b65f70d14e68036a10da92ddb122473

So, the avrdude version you have does not have the 10ms delay workaround for https://savannah.nongnu.org/bugs/?func=detailitem&item_id=42267 at all, so the problem we're seeing is most likely just that. I'm not sure why this commit is reverted, I've created a new issue for that at https://github.com/facchinm/avrdude/issues/13. From here, I think @facchinm must pick this up, there's not more I can do now.

If you want to test this yourself, you could try building avrdude yourself using https://github.com/arduino/avrdude-build-script and/or https://github.com/facchinm/avrdude with https://github.com/facchinm/avrdude/commit/6b0863177b65f70d14e68036a10da92ddb122473 reverted, but I'm not sure how easy that is under Windows (probably needs cygwin or mingw or so).

[kristofmulier]

Hi @matthijskooijman , It's super you found all of this! I have no idea how I would ever find a fix without your help 👍

Mr. @facchinm (https://github.com/facchinm) is the one in charge of the avrdude fork for Arduino IDE?

[facchinm]

Hi everyone, sorry for being a bit late to the party :slightly_smiling_face: I dig a bit through history and the patch was reverted to fix a bug in the Yun build of avrdude (it was the only place using it since avrdude-build-script picked up that git repo only two years later with https://github.com/arduino/avrdude-build-script/commit/cd927feca1567b23180ead889d65e104c2adca5a). Ancient stuff, we should definitely re-revert it. Said that, I also noticed that that avrdude fork is a bit of a mess because it contains the patches for UPDI support from before they were officially merged... A bit of cleanup (and a better commit merge strategy to make history clear) would be needed. I think I can pick it up in a couple of days

[matthijskooijman]

A bit of cleanup (and a better commit merge strategy to make history clear) would be needed. I think I can pick it up in a couple of days

Yay for cleanup! Thanks!

[kristofmulier]

Hi @facchinm , How nice of you to do all of this :-)

Are you the avrdude Arduino-fork manager?

[facchinm]

@kristofmulier unfortunately yes (at the moment) :smile: I'll move the repo to arduino organization as soon as I finish the cleanup

[facchinm]

mainline branch of https://github.com/facchinm/avrdude should be fine now (some bits of information were added to https://github.com/facchinm/avrdude/wiki too).

@matthijskooijman I adopted your version of fuse check patch here https://github.com/facchinm/avrdude/commit/41988d01cd5fd794c7c8dc86f78601183a7e81a1 (see https://github.com/arduino/avrdude-build-script/issues/2#issuecomment-751838160) :wink:

[kristofmulier]

Thank you @facchinm and @matthijskooijman for your help!

When will the patched avrdude be part of the Arduino download? I'm asking because this is my current workflow:

1. I download Arduino on both Windows and Linux.
2. I extract the avrdude tool from the Arduino software.
3. I put the avrdude tool on our Embeetle server.
4. Embeetle IDE users can download it automatically into Embeetle when they need the tool (eg. when they use Arduino boards in Embeetle IDE).

If the patched avrdude will be part of the Arduino download soon, then I don't need to change this workflow. Otherwise, I'd have to find avrdude binaries for Windows and Linux elsewhere, or build them myself.

Kind regards, Kristof

[facchinm]

Hi @kristofmulier , I built avrdude from the new cleaned up sources and published it here temporarily. Still didn't have the time to test it on all operating systems though, just on Linux.

@matthijskooijman I had to replace your "fuses" patch with the old one since it was segfaulting on atmega4809 + edgb (uno wifi rev2). I'm attaching it here alongside the old one as a reference (I'd love to restore it once it's fixed)

patches.zip

[matthijskooijman]

@matthijskooijman I had to replace your "fuses" patch with the old one since it was segfaulting on atmega4809 + edgb (uno wifi rev2). I'm attaching it here alongside the old one as a reference (I'd love to restore it once it's fixed)

Hm, bummer. Any chance you could run it through gdb to get a backtrace? I don't have that hardware, so I don't think I can try that myself...

[facchinm]

Sure, here we go

Writing | ################################################## | 100% 0.02s

avrdude: 1 bytes of fuse5 written
avrdude: verifying fuse5 memory against 0xC9:
avrdude: load data fuse5 data from input file 0xC9:
avrdude: input file 0xC9 contains 1 bytes
avrdude: reading on-chip fuse5 data:

Reading | ################################################## | 100% 0.02s

avrdude: verifying ...

Thread 1 "avrdude" hit Breakpoint 1, 0x000055555555ed50 in get_fuse_bitmask (
    m=0x55555595ccf0) at avr.c:1154
1154              avrdude_message(MSG_INFO, "%s: WARNING: ignoring mismatch in unused bits of \"%s\"\n"
(gdb) c
Continuing.

Thread 1 "avrdude" hit Breakpoint 1, get_fuse_bitmask (
    m=m@entry=0x55555595ccf0) at avr.c:1077
1077      for (i=0; i<32; i++) {
(gdb) c
Continuing.

Thread 1 "avrdude" received signal SIGSEGV, Segmentation fault.
get_fuse_bitmask (m=m@entry=0x55555595ccf0) at avr.c:1078
1078        if (m->op[AVR_OP_WRITE]->bit[i].type == AVR_CMDBIT_INPUT)
(gdb) bt
#0  get_fuse_bitmask (m=m@entry=0x55555595ccf0) at avr.c:1078
#1  0x000055555555ed67 in get_fuse_bitmask (m=0x55555595ccf0) at avr.c:1154
#2  avr_verify (p=p@entry=0x55555595b240, v=v@entry=0x5555559d6670, 
    memtype=0x5555555cd8f0 "fuse5", size=size@entry=1) at avr.c:1136
#3  0x0000555555575db1 in do_op (pgm=0x555555697230, p=p@entry=0x55555595b240, 
    upd=0x5555555cd8c0, flags=flags@entry=UF_NONE) at update.c:361
#4  0x000055555555ac2f in main (argc=<optimized out>, argv=<optimized out>)
    at main.c:1265
(gdb) r

( m->op[AVR_OP_WRITE] is a null pointer)

[kristofmulier]

Hi @facchinm and @matthijskooijman , My apologies for the late response. I just tried again to flash the bootloader on my Arduino UNO with the Atmel ICE and it failed. So I remembered this post and tried the release from @facchinm at https://github.com/arduino/avrdude-build-script/releases/tag/6.3.0-arduino19 (he released it on Jan 8 this year). It works! Below I give some context how exactly I use avrdude.

1. How I use avrdude

I'm part of a team creating a new (free) IDE for microcontrollers: Embeetle IDE (see https://embeetle.com). We're currently working on Arduino support in the IDE. To use avrdude - and actually the whole gnu-avr-toolchain - I copy the following folder from the Arduino installation into Embeetle:

As you can see in the figure above, it's the folder 'C:/Program Files (x86)/Arduino/hardware/tools/avr' is copied into the Embeetle installation folder and renamed as 'gnu_avr_toolchain_7.3.0_32b'. The avrdude.exe executable can then be found here:

In Embeetle IDE, I then invoke avrdude like so:

C:/embeetle/gnu_avr_toolchain_7.3.0_32b/bin/avrdude -C "C:/embeetle/gnu_avr_toolchain_7.3.0_32b/etc/avrdude.conf"
                                                    -v -patmega328p
                                                    -catmelice_isp
                                                    -Pusb
                                                    -e
                                                    -Ulock:w:0x3F:m
                                                    -Uefuse:w:0xFD:m
                                                    -Uhfuse:w:0xDE:m
                                                    -Ulfuse:w:0xFF:m 

and like this for the second command:

C:/embeetle/gnu_avr_toolchain_7.3.0_32b/bin/avrdude -C "C:/embeetle/gnu_avr_toolchain_7.3.0_32b/etc/avrdude.conf"
                                                    -v
                                                    -patmega328p
                                                    -catmelice_isp
                                                    -Pusb
                                                    -Uflash:w:../config/bootloaders/optiboot_atmega328.hex:i
                                                    -Ulock:w:0xCF:m

This second command always ended in the following error, both on the Arduino IDE and in Embeetle IDE:

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
         0xff != 0x0f
avrdude: verification error; content mismatch

avrdude done.  Thank you.

2. Replace avrdude with new build

I just downloaded the latest build (8 Jan 2021) at https://github.com/arduino/avrdude-build-script/releases/tag/6.3.0-arduino19 and unzipped it. Then I did something rather unconventional. I ignored everything in the build and went straight for avrdude.exe - pretending it to be a standalone executable. I then copied it from the unzipped folder into the correct place in Embeetle to replace the avrdude.exe there:

I then launched Embeetle IDE and attempted to flash the bootloader - just like I did before. Now it works. The second command ends in:

avrdude: verifying ...
avrdude: 1 bytes of lock verified

avrdude done.  Thank you.

I'm still a bit flabbergasted that it actually worked. After all, I ignored all files in the build from @facchinm except for avrdude.exe. Is this a standalone executable? Or is it working thanks to the other stuff in gnu_avr_toolchain_7.3.0_32b?

Will this approach also work on Linux? I will try that tomorrow.

Thanks a lot for this new build!

Kind regards, Kristof Mulier

[dbychkov]

Hi. I've tried the AVRDUDE.EXE binary from the recent nightly Arduino build and it still has an error when I try to burn bootloader with Atmel ICE. The facchinm's temporary version works just fine but I would like the fix to be included in the official Arduino build.

[kristofmulier]

Hi @dbychkov , Is this you, Stoyan Shopov?

Kind regards, Kristof

[kristofmulier]

Hi @dbychkov , Sorry for the confusion. Your previous post triggered an email from notifications@github.com, and my e-mail software replaced that address with the name "Stoyan Shopov" in the header, who is a friend of mine.

My apologies.

[facchinm]

@kristofmulier @matthijskooijman I just published the 6.3.0-arduino19 built for all OSes to the downloads server. I'm copying here the links in case you could test them; they'll obviously go through a longer testing phase when enter staging but I'd love to get some first impression on the builds.

        {
          "name": "avrdude",
          "version": "6.3.0-arduino19",
          "systems": [
            {
              "size": "246854",
              "checksum": "SHA-256:598ca19ae60b125f942aaa0a4ff86b1f18c87882ab8edddf8c62fe7717db206c",
              "host": "arm-linux-gnueabihf",
              "archiveFileName": "avrdude-6.3.0-arduino19-armhf-pc-linux-gnu.tar.bz2",
              "url": "http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino19-armhf-pc-linux-gnu.tar.bz2"
            },
            {
              "size": "264179",
              "checksum": "SHA-256:a488228e3cbee5054543f92342266270bb12f181ed4ffdc48ee636be3c36fc6b",
              "host": "aarch64-linux-gnu",
              "archiveFileName": "avrdude-6.3.0-arduino19-aarch64-pc-linux-gnu.tar.bz2",
              "url": "http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino19-aarch64-pc-linux-gnu.tar.bz2"
            },
            {
              "size": "314248",
              "checksum": "SHA-256:952467208571ec0ea0448de1326b52279b6ed42949bd20c0da4d0a29846264f6",
              "host": "x86_64-apple-darwin12",
              "archiveFileName": "avrdude-6.3.0-arduino19-x86_64-apple-darwin12.tar.bz2",
              "url": "http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino19-x86_64-apple-darwin12.tar.bz2"
            },
            {
              "size": "287193",
              "checksum": "SHA-256:b2d60f14b00f74e44a4b241bf0d2c05b759164337fb31f7df6d4ffe662a34aaa",
              "host": "x86_64-linux-gnu",
              "archiveFileName": "avrdude-6.3.0-arduino19-x86_64-pc-linux-gnu.tar.bz2",
              "url": "http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino19-x86_64-pc-linux-gnu.tar.bz2"
            },
            {
              "size": "278384",
              "checksum": "SHA-256:eb9d6b423f351c4da3693f51ff7906e7bbb1b53ef70055fc4ff11604e781fbfd",
              "host": "i686-linux-gnu",
              "archiveFileName": "avrdude-6.3.0-arduino19-i686-pc-linux-gnu.tar.bz2",
              "url": "http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino19-i686-pc-linux-gnu.tar.bz2"
            },
            {
              "size": "335177",
              "checksum": "SHA-256:81c034c6bb932d4cfd0d14fd1c5455f2bdbf41ee7ede6c86432a924c82e73256",
              "host": "i686-mingw32",
              "archiveFileName": "avrdude-6.3.0-arduino19-i686-w64-mingw32.zip",
              "url": "http://downloads.arduino.cc/tools/avrdude-6.3.0-arduino19-i686-w64-mingw32.zip"
            }
          ]
        },

[dbychkov]

Hi,

I've copied AVRDUDE files from "avrdude-6.3.0-arduino19-i686-w64-mingw32.zip" to my Arduino install folder and tried to burn bootloader and upload sketch to the Arduino Nano via ISP with Atmel-ICE programmer. Both tests were completed successfully. Attached 2 files with console output from Arduino/AVRDUDE.

OS: Windows 7 Ultimate SP1, x64 on Intel i7 Arduino: 1.8.13 Programmer: Atmel-ICE

Thank you for fixing this issue. burn_bootloader.txt upload_sketch.txt

[matthijskooijman]

I don't have a setup for testing ready quickly, but I'm interested in the source and patches applied to have a look at that side of things. However, it seems that's still a bit unclear, with @facchinm's avrdude repo being used without being tagged, so it's not quite clear which code is used to build each version exactly. Maybe that's something to clear up, i.e. using git submodules or git tags in the https://github.com/facchinm/avrdude repository? MIght be better to discuss in https://github.com/arduino/avrdude-build-script/issues/15, though.