arduino / avrdude-packing

This repository contains the release pipeline to build statically avrdude
6 stars 1 forks source link

libusb0.dll missing #15

Closed kristofmulier closed 1 year ago

kristofmulier commented 2 years ago

libusb0.dll missing

At Embeetle IDE, we provide one of your Windows releases on our download server. One of our users experienced the following error on Windows 10 when trying to run the executable:

image

The error message reads:

avrdude.exe - System Error The code execution cannot proceed because libusb0.dll was not found. Reinstalling the program may fix this problem.

 

AVRDUDE version

The user downloaded the following AVRDUDE version: https://new.embeetle/downloads/beetle_tools/windows/avrdude_7.0.0_32b.7z

It's basically one of your 7.0 releases. I don't remember if it's 7.0-arduino.1, 7.0-arduino.2-rc1 or 7.0-arduino.2 (it certainly isn't 7.0-arduino.3 yet). When I run the executable on my PC, I only get this version output:

avrdude version 7.0, URL: <https://github.com/avrdudes/avrdude>

So I know it's version 7.0 - but not more specific.

Notes

mariusgreuel commented 2 years ago

@kristofmulier It seems like the Arduino release uses libusb0. May I suggest that you use the official MSVC avrdude releases instead, or the MSVC build procedure if you wish to build it yourself?

The Windows MSVC build is very different from the MinGW build, as it links against a rewritten libusb and ftdi library. I designed these libraries to fix some of the most common problems of the previous releases, which is the lack of native WinUSB and FTDI driver support. If you link against libusb0, that means your release it lacking WinUSB support, which is terrible for Windows users, because many devices will not work with a WinUSB based plug-and-play driver.

Details here: https://github.com/avrdudes/avrdude/wiki/The-story-on-libusb-for-Windows

In short, we should not enforce the use of outdated driver technology such as libusb0.sys by not supporting WinUSB.

I would appreciate it very much if a major distribution such as Arduino would pick up the official Windows release from https://github.com/avrdudes/avrdude/releases/tag/v7.0 or alternatively, use the msvc workflow from https://github.com/avrdudes/avrdude/blob/main/.github/workflows/build.yml

kristofmulier commented 2 years ago

@mariusgreuel , So I should take the Windows binaries from: https://github.com/avrdudes/avrdude/releases/tag/v7.0

and the Linux binaries from: https://github.com/arduino/avrdude-packing/releases/tag/7.0-arduino.3

(because the official avrdude repo has no Linux binaries yet)

Correct?

umbynos commented 2 years ago

I did not know about this story. For sure, we will try to solve the problem. Thanks for the detailed report, and for the explanation.

mariusgreuel commented 2 years ago

So I should take the Windows binaries from

Yes.

Didn't you link the libusb library statically? Why is AVRDUDE looking around for the dll?

Even if you link statically to libusb-0.1, the library introduces a DLL dependency to libusb0.dll. That DLL ships with the libusb0.sys driver package, I guess with the intention that there is another layer in between, which allows for breaking changes in the libusb0.sys driver. However, the design flaw is that libusb-0.1 has a static dependency to libusb0.dll - if the driver package is not installed (because it is not needed), you get the error message you attached. The fix would have been to load libusb0.dll dynamically.

To workaround that problem, everybody ships libusb0.dll as a side-by-side DLL with avrdude - thus breaking the idea of being able to update that DLL via the driver package.

So, the Arduino release links libusb-0.1 correctly - it is your responsibility to ship libusb0.dll as well for those people who do not have the libusb0.sys driver package installed.

Note that if you choose to use the MSVC release of avrdude, you should NOT ship libusb0.dll. Instead, you can rely on the libusb0.sys driver package to install a matching libusb0.dll.

kristofmulier commented 2 years ago

Note that if you choose to use the MSVC release of avrdude, you should NOT ship libusb0.dll. Instead, you can rely on the libusb0.sys driver package to install a matching libusb0.dll.

Great to hear that libusb0.sys does this automatically. Is that libusb0.sys driver present on every Windows by default? Or does the user need to install it manually?

PS: My knowledge about libusb is very limited. Forgive me if my question sounds stupid.

mcuee commented 2 years ago

WinUSB driver is the way to go. libusb0.sys (or libusbk.sys) is not available by default under Windows -- you usually use Zadig (or libusbk-inf-wizard) to instal the driver which will then install libusb0.dll/libusbk.dll for you as well.

Best way is for the HW vendors to go with WCID, then you do not need to worry about driver installation for newer version of Windows. https://github.com/pbatard/libwdi/wiki/WCID-Devices

There are good alternative FW to offer WCID compatible FW for USBASP. My favorite is this one but it does not work with official avrdude 7.0 Windows binary due to https://github.com/avrdudes/avrdude/issues/968 https://github.com/dioannidis/usbasp

The following WCID compliant FW is also pretty good and should work with the official avrdude 7.0 binary. https://github.com/nerdralph/usbasp

BTW, quite some Microchip AVR tools now use WinUSB as the driver, for example, the popular AVR ISP mkii.

mcuee commented 2 years ago

There is no static mingw-compatible libary for libusb-win32 in reality. libusb-win32 project and libusbk project do not support building the driver or static library using mingw. The only thing we support is the way to dynamically import the DDK/MSVC dll to use with MinGW.

mariusgreuel commented 2 years ago

Is that libusb0.sys driver present on every Windows by default?

It is neither a Windows provided driver, nor a Plug-And-Play driver (i.e. provided through the Windows Update Catalog). It can only be installed manually by the user or by third-party applications.

Hence, the author lost total control over his ability to update the driver. As there has not been any updates in years, I would think it is safe to assume that millions of computers are left vulnerable through this unmanaged driver package. Just one of the many reasons not to distribute (or promote distribution of) libusb0.sys.

mcuee commented 2 years ago

Even though I am one of the admins of libusb-win32 project, I do not really recommend to use it as it is in maintainence mode. Ref: https://github.com/mcuee/libusb-win32/wiki

libusb-win32 is now in Bug-Fix-Only maintenance mode. It is recommended new project should migrate to libusb-1.0 API and use libusb Windows (http://libusb.info/) instead. For Windows only project, you can also look at libusbK below (http://libusbk.sourceforge.net/UsbK3/index.html).

That being said, we have not declared libusb-win32 project dead yet as there are still many users. The current developer (Peter Dons Tychsen) has fixed a few issues in libusb-win32 1.2.6.0 release and come out with release 1.2.7.3. It is marked as snapshot release due to missing the driver installers and some updates (like porting to new WDK). It should be a good upgrade for the existing users who still want to keep using libsub-win32 for various reasons. Example: Microchip FLIP still uses libusb-win32 and have not upgraded to use WinUSB. Example: there are legacy devices which do not work with WinUSB due to FW issues.

Update history: https://github.com/mcuee/libusb-win32/commits/master https://github.com/mcuee/libusb-win32/releases

1.2.7.3 is the default download in libusb-win32 Sourceforge site and I can see there are more downloads of 1.2.7.3 version compared to 1.2.6.0 version. https://sourceforge.net/projects/libusb-win32/files/

mcuee commented 2 years ago

@umbynos As I know you are using cross-compile and MinGW, the simple fix is just to ship the libusb0.dll with your binary.

The other solution is to switch to libusb-compat-0.1 which sits on top of libusb-1.0. With libusb-1.0.26 release, libusb0.sys can be used without issues, at least for the purpose of avrdude/Arduino.

Edit to update: if you follow the instructions in the official avrdude wiki, you will actually use the second option and the result binary will actually not depend on libusb0.dll (all libusb-win32 library). Same if you use build.sh under MSYS2 mingw32/64. https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-MSYS2

My personal avrdude MSYS2 mingw64 build here are all using the second option (to use libusb-1.0 + libusb-compat-0.1). https://github.com/mcuee/avrdude/releases/tag/v7.0

umbynos commented 2 years ago

@mcuee thanks for the suggestion!

kristofmulier commented 2 years ago

[@mcuee] As I know you are using cross-compile and MinGW, the simple fix is just to ship the libusb0.dll with your binary.

This would indeed fix a few issues, but we should remember what @mariusgreuel said:

[@mariusgreuel] The Windows MSVC build is very different from the MinGW build, as it links against a rewritten libusb and ftdi library. I designed these libraries to fix some of the most common problems of the previous releases, which is the lack of native WinUSB and FTDI driver support. If you link against libusb0, that means you release it lacking WinUSB support, which is terrible for Windows users, because many devices will not work with a WinUSB based plug-and-play driver.

Native WinUSB and FTDI driver support sounds like a must-have to me. Therefore, I believe it's best to follow the advice from @mariusgreuel and build the Windows version as he suggests. What do you think @umbynos?

mcuee commented 2 years ago

Native WinUSB and FTDI driver support sounds like a must-have to me. Therefore, I believe it's best to follow the advice from @mariusgreuel and build the Windows version as he suggests. What do you think @umbynos?

@kristofmulier Native WinUSB support is available without using the MSVC build -- you can use the mingw build as well. The main benefit of using avrdude-libwinusb is now moot since libusb-1.0.26 has fixed most of the issues with libusb0.sys support and I do not see any issues using libusb-1.0.26 + libusb-compat-0.1 to work with devices (either libusb0.sys or WinUSB driver, or using libusbk driver). You do not need to use libusb-win32 at all if you do not like it.

And avrdude-libwinusb may have other issues other than the lack of support for USB composite device since it is rather new. You can refer to the comments by @mariusgreuel himself. That being said, I have not encountered issues myself other than the above USB composite Device support issue. Still I do not use the MSVC build often as I prefer to use MSYS2 mingw64.

avrdude-libftdi does have benefits compared to libftdi-1.0 (or libftdi-0.1) as you do not need to swap out the FTDI vendor driver. That being said, there are licensing concerns. So I am not so sure if @umbynos will accept that. Persoanlly I believe the approach by @mariusgreuel is acceptable though (avrdude-libftdi is kind of a wrapper around D2XX DLL). But I've seen heated debate in the openocd project many years ago which resulted in the removal of D2XX support (and the exit of the original developer and a few others).

kristofmulier commented 2 years ago

Unfortunately, I'm a complete noob in this topic, so my input is limited. Regardless of the build decisions you take - I (and many other AVRDUDE users) just need a single webpage to download all the latest binaries from: both for Windows and Linux. The binaries should just work, even if the user misses a few dll's here and there.

I know this sounds simple, but it is very hard to achieve. Software distribution is damn hard - we experience it first-hand ourselves with Embeetle IDE. Therefore I express my sincere gratitude to each of you involved in the distribution of AVRDUDE @mcuee , @mariusgreuel and @umbynos !

umbynos commented 2 years ago

The other solution is to switch to libusb-compat-0.1 which sits on top of libusb-1.0. With libusb-1.0.26 release, libusb0.sys can be used without issues, at least for the purpose of avrdude/Arduino.

I think we started using libusb-compat-0.1.7 and libusb-1.0.26 from release 7.0-arduino.2 -> see here and here. @mcuee Am I missing something?

Should I include the lib here using -DCMAKE_C_FLAGS?

mcuee commented 2 years ago

@umbynos Sorry I am not an expert for CMake-- my only guess is that you need to remove libusb-win32 from your cross-build environment. It could be that both libusb-compat-0.1 and libusb-win32 are both presented in your cross-build environment and somehow libusb-win32 was picked (espcially if the header file name is changed to usb.h from the original lusb_usb0.h.

mcuee commented 2 years ago

I use a simple tool to check the dll dependency of the windows avrdude.exe (there are many other tools, depenancy walker is the most famous but does not seem to work with my Windows 11 laptop).

CFE Explorer: https://ntcore.com/?page_id=388

Results for the the binary from this repo (https://github.com/arduino/avrdude-packing/releases/download/7.0-arduino.3/avrdude_7.0-arduino.3_Windows_32bit.tar.gz) dll_import

mcuee commented 2 years ago

Results for the mingw32 build from avrdude github action. https://github.com/avrdudes/avrdude/actions/runs/2798452821 (latest git main commit) https://github.com/avrdudes/avrdude/suites/7670353048/artifacts/319829314

dll_import_avrdude_githubaction

mcuee commented 2 years ago

@umbynos If you compared the two, you will see that somehow the Arduino binary adds dependency to libusb0.dll but lack the import of hid.dll.

As of now, hid.dll is needed for PICKit 2 support for Windows users, which may not be a real issue for most of the Arduino users (this should really be ported to hidapi). Ref: https://github.com/avrdudes/avrdude/issues/883

BTW, PICKit 2 is currently not supported by avrdude under macOS because it has not been ported to hidapi.

umbynos commented 2 years ago

Ok, I think I've found why it's including that dep: https://github.com/arduino/crossbuild/blob/f1e2bb941634c461c94f3f7bad10cd00ff7e775c/deps/build_libs.sh#L25-L33 I've used libusb-win32 because apparently "libusb-compat was a mess to compile for win32". At this point I think the only thing left is to compile natively the win binary on win using MSVC or MSYS2, even if I don't like to have two different build systems.

mcuee commented 2 years ago

@umbynos FYI, libusb-compat-0.1 is not that difficult to compile for Windows. MSYS2 has the formula. I think you can still cross-compile it. https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-libusb-compat-git/PKGBUILD

Personally I like to use native build though (for projects like: libusb, libftdi, hidapi, openocd and avrdude) -- usually using MSYS2 mingw32/64, occassionaly using MSVC.

umbynos commented 2 years ago

I could have some news: I tried to use libusb-compat instead of libusb-win32. I'm also updating ncurses from 5.9 to 6.3 in the build container. After some attempt this is the resulting binary avrdude-main-test-libusb-compat.zip It's build from https://github.com/avrdudes/avrdude/commit/a1018999c4482dc5e28620d8ad932b57c4fd4c57 and has only 3 patches applied:

Please @mcuee could you help me in checking if the deps are correct and if the binary is working correcly? :wrench:

mcuee commented 2 years ago

Please @mcuee could you help me in checking if the deps are correct and if the binary is working correcly?

@umbynos Yes this works well. Thanks.

/c/work/avr/avrdude_test/avrdude_binary_archive/avrdude-main-test-libusb-compat
$ ldd ./avrdude.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7fffe2f60000)
        ntdll.dll => /c/Windows/SysWOW64/ntdll.dll (0x775d0000)
        wow64.dll => /c/WINDOWS/System32/wow64.dll (0x7fffe2cf0000)
        wow64base.dll => /c/WINDOWS/System32/wow64base.dll (0x7fffe1410000)
        wow64win.dll => /c/WINDOWS/System32/wow64win.dll (0x7fffe1620000)
        wow64con.dll => /c/WINDOWS/System32/wow64con.dll (0x7fffe2120000)

The output is similar to what I get from mingw32 static link version of avrdude git main, and avrdude 7.0 release msvc build.

$ ldd avrdude_mingw32_git.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ff89c1c0000)
        ntdll.dll => /c/Windows/SysWOW64/ntdll.dll (0x778f0000)
        wow64.dll => /c/WINDOWS/System32/wow64.dll (0x7ff89b300000)
        wow64base.dll => /c/WINDOWS/System32/wow64base.dll (0x7ff89a160000)
        wow64win.dll => /c/WINDOWS/System32/wow64win.dll (0x7ff89bea0000)
        wow64con.dll => /c/WINDOWS/System32/wow64con.dll (0x7ff89b2e0000)

$ ldd avrdude7_msvc32.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ff89c1c0000)
        ntdll.dll => /c/Windows/SysWOW64/ntdll.dll (0x778f0000)
        wow64.dll => /c/WINDOWS/System32/wow64.dll (0x7ff89b300000)
        wow64base.dll => /c/WINDOWS/System32/wow64base.dll (0x7ff89a160000)
        wow64win.dll => /c/WINDOWS/System32/wow64win.dll (0x7ff89bea0000)
        wow64con.dll => /c/WINDOWS/System32/wow64con.dll (0x7ff89b2e0000)

Screenshot_avrdude_arduino_test

mcuee commented 2 years ago

@umbynos

I also tested with usbasp using all three different drivers (libusbk, libusb0.sys and WinUSB) and all three work fine.

PS C:\work\avr\avrdude_test\avrdude_binary_archive\avrdude-main-test-libusb-compat> .\kList.exe

Loading USB ID's maintained by Stephen J. Gowdy <linux.usb.ids@gmail.com>..

 1. USBasp (VOTI) [Connected]
    Service              : libusb0
    ClassGUID            : {EB781AAF-9C70-4523-A5DF-642A87ECA567}
    DeviceID             : USB\VID_16C0&PID_05DC\0001
    DeviceInterfaceGUID  : {B3F4C5CA-2A80-9D6A-1A90-D7BADF35524F}
    SymbolicLink         : \\?\usb#vid_16c0&pid_05dc#0001#{b3f4c5ca-2a80-9d6a-1a90-d7badf35524f}
    DevicePath           : \\.\libusb0-0001
    SerialNumber         : 0001
    BusNumber            : 0
    DeviceAddress        : 1

Select device (1-1) :
invalid selection

PS C:\work\avr\avrdude_test\avrdude_binary_archive\avrdude-main-test-libusb-compat> 
.\avrdude -c usbasp -p m328p -qq -U .\Blink.ino.with_bootloader.standard.hex && echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_binary_archive\avrdude-main-test-libusb-compat> .\kList.exe

Loading USB ID's maintained by Stephen J. Gowdy <linux.usb.ids@gmail.com>..

 1. USBasp (VOTI) [Connected]
    Service              : libusbK
    ClassGUID            : {ECFB0CFD-74C4-4F52-BBF7-343461CD72AC}
    DeviceID             : USB\VID_16C0&PID_05DC\0001
    DeviceInterfaceGUID  : {D32EDB66-69FC-01DC-03DD-20936E7318C2}
    SymbolicLink         : \\?\usb#vid_16c0&pid_05dc#0001#{d32edb66-69fc-01dc-03dd-20936e7318c2}
    DevicePath           : \\?\usb#vid_16c0&pid_05dc#0001#{d32edb66-69fc-01dc-03dd-20936e7318c2}
    SerialNumber         : 0001
    BusNumber            : 0
    DeviceAddress        : 1

Select device (1-1) :
invalid selection

PS C:\work\avr\avrdude_test\avrdude_binary_archive\avrdude-main-test-libusb-compat> 
.\avrdude -c usbasp -p m328p -qq -U .\Blink.ino.with_bootloader.standard.hex && echo OK
OK

PS C:\work\avr\avrdude_test\avrdude_binary_archive\avrdude-main-test-libusb-compat> .\kList.exe

Loading USB ID's maintained by Stephen J. Gowdy <linux.usb.ids@gmail.com>..

 1. USBasp (VOTI) [Connected]
    Service              : libusbK
    ClassGUID            : {ECFB0CFD-74C4-4F52-BBF7-343461CD72AC}
    DeviceID             : USB\VID_16C0&PID_05DC\0001
    DeviceInterfaceGUID  : {D32EDB66-69FC-01DC-03DD-20936E7318C2}
    SymbolicLink         : \\?\usb#vid_16c0&pid_05dc#0001#{d32edb66-69fc-01dc-03dd-20936e7318c2}
    DevicePath           : \\?\usb#vid_16c0&pid_05dc#0001#{d32edb66-69fc-01dc-03dd-20936e7318c2}
    SerialNumber         : 0001
    BusNumber            : 0
    DeviceAddress        : 1

Select device (1-1) :
invalid selection

PS C:\work\avr\avrdude_test\avrdude_binary_archive\avrdude-main-test-libusb-compat> 
.\avrdude -c usbasp -p m328p -qq -U .\Blink.ino.with_bootloader.standard.hex && echo OK
OK
mcuee commented 2 years ago

@umbynos

Could you also attach the Linux and macOS build? I can test them as well. Thanks.

BTW, the container works well with avrdude 7.0 release and avrdude git main 32bit Linux i686 build as well as macOS x86_64 build. But I'd like to test your build as the build flags may not be exactly the same. And I think you have a more up-to-date container than the published one.

mcuee commented 2 years ago

I do not know how to set up Arduino to use this new version of avrdude. So I just overwrite the official 6.3 version with the new version. It seems to work fine as well, tested using @MCUdude's MiniCore with Arduino 2.01 under Windows 11.

"C:\Users\xiaof\AppData\Local\Arduino15\packages\arduino\tools\avrdude\6.3.0-arduino18/bin/avrdude" "-CC:\Users\xiaof\AppData\Local\Arduino15\packages\MiniCore\hardware\avr\2.1.3/avrdude.conf" -v -V -patmega168p -carduino -PCOM3 -b115200 -D "-Uflash:w:C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.hex:i"

avrdude: Version avrdude-main-test-libusb-compat
         Copyright (c) Brian Dean, http://www.bdmicro.com/
         Copyright (c) Joerg Wunsch

         System wide configuration file is C:\Users\xiaof\AppData\Local\Arduino15\packages\MiniCore\hardware\avr\2.1.3\avrdude.conf

         Using Port                    : COM3
         Using Programmer              : arduino
         Overriding Baud Rate          : 115200
         AVR Part                      : ATmega168P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         Serial program mode           : yes
         Parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                           Block Poll               Page                       Polled
           Memory Type Alias    Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom                 65    20     4    0 no        512    4      0  3600  3600 0xff 0xff
           flash                  65     6   128    0 yes     16384  128    128  4500  4500 0xff 0xff
           lfuse                   0     0     0    0 no          1    1      0  4500  4500 0x00 0x00
           hfuse                   0     0     0    0 no          1    1      0  4500  4500 0x00 0x00
           efuse                   0     0     0    0 no          1    1      0  4500  4500 0x00 0x00
           lock                    0     0     0    0 no          1    1      0  4500  4500 0x00 0x00
           calibration             0     0     0    0 no          1    1      0     0     0 0x00 0x00
           signature               0     0     0    0 no          3    1      0     0     0 0x00 0x00

         Programmer Type : Arduino
         Description     : Arduino
         Hardware Version: 3
         Firmware Version: 8.0

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e940b (probably m168p)
avrdude: reading input file C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.hex for flash
         with 1030 bytes in 1 section within [0, 0x405]
         using 9 pages and 122 pad bytes
avrdude: writing 1030 bytes flash ...

Writing | ################################################## | 100% 0.22s

avrdude: 1030 bytes of flash written

avrdude done.  Thank you.

It is also good with MegaCoreX after I overwrite the default avrdude.conf file.

"C:\Users\xiaof\AppData\Local\Arduino15\packages\MegaCoreX\tools\avrdude\7.0.0-arduino2/bin/avrdude" "-CC:\Users\xiaof\AppData\Local\Arduino15\packages\MegaCoreX\hardware\megaavr\1.1.0/avrdude.conf" -v -V -patmega4808 -cjtag2updi -PCOM13 "-Uflash:w:C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.with_bootloader.hex:i" "-Ufuse0:w:0x00:m" "-Ufuse1:w:0x54:m" "-Ufuse2:w:0x01:m" "-Ufuse4:w:0x00:m" "-Ufuse5:w:0b11001001:m" "-Ufuse6:w:0x06:m" "-Ufuse7:w:0x00:m" "-Ufuse8:w:0x00:m" "-Ulock:w:0xC5:m"

avrdude: Version avrdude-main-test-libusb-compat
         Copyright (c) Brian Dean, http://www.bdmicro.com/
         Copyright (c) Joerg Wunsch

         System wide configuration file is C:\Users\xiaof\AppData\Local\Arduino15\packages\MegaCoreX\hardware\megaavr\1.1.0\avrdude.conf

         Using Port                    : COM13
         Using Programmer              : jtag2updi
JTAG ICE mkII sign-on message:
Communications protocol version: 1
M_MCU:
  boot-loader FW version:        1
  firmware version:              6.00
  hardware version:              1
S_MCU:
  boot-loader FW version:        1
  firmware version:              6.00
  hardware version:              1
Serial number:                   0a:0b:0c:0d:0e:0f
Device ID:                       Thinary mkII
         AVR Part                      : ATmega4808
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         Serial program mode           : yes
         Parallel program mode         : yes
         Memory Detail                 :

                                           Block Poll               Page                       Polled
           Memory Type Alias    Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           fuse0       wdtcfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse1       bodcfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse2       osccfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse4       tcd0cfg     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse5       syscfg0     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse6       syscfg1     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse7       append      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse8       bootend     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuses                   0     0     0    0 no          9   10      0     0     0 0x00 0x00
           lock                    0     0     0    0 no          1    1      0     0     0 0x00 0x00
           tempsense               0     0     0    0 no          2    1      0     0     0 0x00 0x00
           signature               0     0     0    0 no          3    1      0     0     0 0x00 0x00
           prodsig                 0     0     0    0 no         61   61      0     0     0 0x00 0x00
           sernum                  0     0     0    0 no         10    1      0     0     0 0x00 0x00
           osccal16                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           osccal20                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           osc16err                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           osc20err                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           data                    0     0     0    0 no          0    1      0     0     0 0x00 0x00
           userrow     usersig     0     0     0    0 no         64   64      0     0     0 0x00 0x00
           eeprom                  0     0     0    0 no        256   64      0     0     0 0x00 0x00
           flash                   0     0     0    0 no      49152  128      0     0     0 0x00 0x00

         Programmer Type : JTAGMKII_UPDI
         Description     : JTAGv2 to UPDI bridge
         M_MCU HW version: 1
         M_MCU FW version: 6.00
         S_MCU HW version: 1
         S_MCU FW version: 6.00
         Serial number   : 0a:0b:0c:0d:0e:0f
         Vtarget         : 5.0 V

avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9650 (probably m4808)
avrdude: Note: flash memory has been specified, an erase cycle will be performed.
         To disable this feature, specify the -D option.
erasing chip
avrdude: reading input file C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.with_bootloader.hex for flash
         with 758 bytes in 1 section within [0, 0x2f5]
         using 6 pages and 10 pad bytes
avrdude: writing 758 bytes flash ...

Writing | ################################################## | 100% 0.21s

avrdude: 758 bytes of flash written
avrdude: reading input file 0x00 for fuse0/wdtcfg
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse0/wdtcfg ...
avrdude: 1 byte of fuse0/wdtcfg written
avrdude: reading input file 0x54 for fuse1/bodcfg
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse1/bodcfg ...
avrdude: 1 byte of fuse1/bodcfg written
avrdude: reading input file 0x01 for fuse2/osccfg
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse2/osccfg ...
avrdude: 1 byte of fuse2/osccfg written
avrdude: reading input file 0x00 for fuse4/tcd0cfg
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse4/tcd0cfg ...
avrdude: 1 byte of fuse4/tcd0cfg written
avrdude: reading input file 0b11001001 for fuse5/syscfg0
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse5/syscfg0 ...
avrdude: 1 byte of fuse5/syscfg0 written
avrdude: reading input file 0x06 for fuse6/syscfg1
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse6/syscfg1 ...
avrdude: 1 byte of fuse6/syscfg1 written
avrdude: reading input file 0x00 for fuse7/append
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse7/append ...
avrdude: 1 byte of fuse7/append written
avrdude: reading input file 0x00 for fuse8/bootend
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse8/bootend ...
avrdude: 1 byte of fuse8/bootend written
avrdude: reading input file 0xC5 for lock
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte lock ...
avrdude: 1 byte of lock written

avrdude done.  Thank you.
mcuee commented 2 years ago

It also works with the megaTinyCore from @SpenceSpenceKonde, after I overwrite the default avrdude.conf file.

"C:\Users\xiaof\AppData\Local\Arduino15\packages\DxCore\tools\avrdude\6.3.0-arduino17or18/bin/avrdude" "-CC:\Users\xiaof\AppData\Local\Arduino15\packages\megaTinyCore\hardware\megaavr\2.6.1/avrdude.conf" -v -V -pattiny817 -cxplainedmini_updi -Pusb "-Ufuse5:w:0b11000101:m" "-Ufuse6:w:0x04:m" "-Ufuse7:w:0x00:m" "-Ufuse8:w:0x00:m" "-Uflash:w:C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.hex:i"

avrdude: Version avrdude-main-test-libusb-compat
         Copyright (c) Brian Dean, http://www.bdmicro.com/
         Copyright (c) Joerg Wunsch

         System wide configuration file is C:\Users\xiaof\AppData\Local\Arduino15\packages\megaTinyCore\hardware\megaavr\2.6.1\avrdude.conf

         Using Port                    : usb
         Using Programmer              : xplainedmini_updi
avrdude: found CMSIS-DAP compliant device, using EDBG protocol
         AVR Part                      : ATtiny817
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         Serial program mode           : yes
         Parallel program mode         : yes
         Memory Detail                 :

                                           Block Poll               Page                       Polled
           Memory Type Alias    Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           fuse0       wdtcfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse1       bodcfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse2       osccfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse4       tcd0cfg     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse5       syscfg0     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse6       syscfg1     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse7       append      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse8       bootend     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuses                   0     0     0    0 no          9   10      0     0     0 0x00 0x00
           lock                    0     0     0    0 no          1    1      0     0     0 0x00 0x00
           tempsense               0     0     0    0 no          2    1      0     0     0 0x00 0x00
           signature               0     0     0    0 no          3    1      0     0     0 0x00 0x00
           prodsig                 0     0     0    0 no         61   61      0     0     0 0x00 0x00
           sernum                  0     0     0    0 no         10    1      0     0     0 0x00 0x00
           osccal16                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           osccal20                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           osc16err                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           osc20err                0     0     0    0 no          2    1      0     0     0 0x00 0x00
           data                    0     0     0    0 no          0    1      0     0     0 0x00 0x00
           userrow     usersig     0     0     0    0 no         32   32      0     0     0 0x00 0x00
           eeprom                  0     0     0    0 no        128   32      0     0     0 0x00 0x00
           flash                   0     0     0    0 no       8192   64      0     0     0 0x00 0x00

         Programmer Type : JTAGICE3_UPDI
         Description     : Atmel AVR XplainedMini in UPDI mode
         ICE HW version  : 0
         ICE FW version  : 1.13 (rel. 43)
         Serial number   : ATML2658061800009257
         Vtarget         : 5.00 V
avrdude: partial Family_ID returned: "tiny"
avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e9320 (probably t817)
avrdude: Note: flash memory has been specified, an erase cycle will be performed.
         To disable this feature, specify the -D option.
erasing chip
avrdude: reading input file 0b11000101 for fuse5/syscfg0
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse5/syscfg0 ...
avrdude: 1 byte of fuse5/syscfg0 written
avrdude: reading input file 0x04 for fuse6/syscfg1
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse6/syscfg1 ...
avrdude: 1 byte of fuse6/syscfg1 written
avrdude: reading input file 0x00 for fuse7/append
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse7/append ...
avrdude: 1 byte of fuse7/append written
avrdude: reading input file 0x00 for fuse8/bootend
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse8/bootend ...
avrdude: 1 byte of fuse8/bootend written
avrdude: reading input file C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.hex for flash
         with 592 bytes in 1 section within [0, 0x24f]
         using 10 pages and 48 pad bytes
avrdude: writing 592 bytes flash ...

Writing | ################################################## | 100% 0.27s

avrdude: 592 bytes of flash written

avrdude done.  Thank you.
mcuee commented 2 years ago

For DxCore, other than overwriting the default avrdude.conf, I also need to update "programmer.txt" to change curiosity_updi to pkobn_updi.

"C:\Users\xiaof\AppData\Local\Arduino15\packages\DxCore\tools\avrdude\6.3.0-arduino17or18/bin/avrdude" "-CC:\Users\xiaof\AppData\Local\Arduino15\packages\DxCore\hardware\megaavr\1.4.10/avrdude.conf" -v -V -pavr128db48 -cpkobn_updi -Pusb "-Ufuse5:w:0b11001001:m" "-Ufuse6:w:0b00001100:m" "-Ufuse7:w:0x00:m" "-Ufuse8:w:0x00:m" "-Uflash:w:C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.hex:i"

avrdude: Version avrdude-main-test-libusb-compat
         Copyright (c) Brian Dean, http://www.bdmicro.com/
         Copyright (c) Joerg Wunsch

         System wide configuration file is C:\Users\xiaof\AppData\Local\Arduino15\packages\DxCore\hardware\megaavr\1.4.10\avrdude.conf

         Using Port                    : usb
         Using Programmer              : pkobn_updi
avrdude: found CMSIS-DAP compliant device, using EDBG protocol
         AVR Part                      : AVR128DB48
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         Serial program mode           : yes
         Parallel program mode         : yes
         Memory Detail                 :

                                           Block Poll               Page                       Polled
           Memory Type Alias    Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           fuse0       wdtcfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse1       bodcfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse2       osccfg      0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse4       tcd0cfg     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse5       syscfg0     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse6       syscfg1     0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse7       codesize    0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuse8       bootsize    0     0     0    0 no          1    1      0     0     0 0x00 0x00
           fuses                   0     0     0    0 no          9   16      0     0     0 0x00 0x00
           lock                    0     0     0    0 no          4    1      0     0     0 0x00 0x00
           tempsense               0     0     0    0 no          2    1      0     0     0 0x00 0x00
           signature               0     0     0    0 no          3    1      0     0     0 0x00 0x00
           prodsig                 0     0     0    0 no        125  125      0     0     0 0x00 0x00
           sernum                  0     0     0    0 no         16    1      0     0     0 0x00 0x00
           userrow     usersig     0     0     0    0 no         32   32      0     0     0 0x00 0x00
           data                    0     0     0    0 no          0    1      0     0     0 0x00 0x00
           eeprom                  0     0     0    0 no        512    1      0     0     0 0x00 0x00
           flash                   0     0     0    0 no     131072  512      0     0     0 0x00 0x00

         Programmer Type : JTAGICE3_UPDI
         Description     : Curiosity nano (nEDBG) in UPDI mode
         ICE HW version  : 0
         ICE FW version  : 1.21 (rel. 37)
         Serial number   : MCHP3372031800001331
         Vtarget         : 3.31 V
         PDI/UPDI clock Xmega/megaAVR : 100 kHz
avrdude: partial Family_ID returned: "AVR "
avrdude: AVR device initialized and ready to accept instructions
avrdude: device signature = 0x1e970c (probably avr128db48)
avrdude: Note: flash memory has been specified, an erase cycle will be performed.
         To disable this feature, specify the -D option.
erasing chip
avrdude: reading input file 0b11001001 for fuse5/syscfg0
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse5/syscfg0 ...
avrdude: 1 byte of fuse5/syscfg0 written
avrdude: reading input file 0b00001100 for fuse6/syscfg1
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse6/syscfg1 ...
avrdude: 1 byte of fuse6/syscfg1 written
avrdude: reading input file 0x00 for fuse7/codesize
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse7/codesize ...
avrdude: 1 byte of fuse7/codesize written
avrdude: reading input file 0x00 for fuse8/bootsize
         with 1 byte in 1 section within [0, 0]
avrdude: writing 1 byte fuse8/bootsize ...
avrdude: 1 byte of fuse8/bootsize written
avrdude: reading input file C:\Users\xiaof\AppData\Local\Temp\arduino-sketch-07DCBE58C31D28F5C44D1058B336EB98/Blink.ino.hex for flash
         with 1008 bytes in 1 section within [0, 0x3ef]
         using 2 pages and 16 pad bytes
avrdude: writing 1008 bytes flash ...

Writing | ################################################## | 100% 0.31s

avrdude: 1008 bytes of flash written

avrdude done.  Thank you.
umbynos commented 1 year ago

Hi @mcuee, I just managed to release a test version of avrdude with the new container, I know, It took me a bit of time... :sweat: It's built with crossbuild v0.2.2:

The base commit is https://github.com/avrdudes/avrdude/commit/4c92030e3a486cfbaeb36e298f6f2929e8e031eb

I'm trying to update everything so that when the new version of avrdude (7.1) comes out we are ready and can build a version that is production ready and that we can start to distribute to our users

umbynos commented 1 year ago

Also, please @kristofmulier if the new 7.0-arduino.4-rc1 fixes the problem could we close this issue? :smile:

MCUdude commented 1 year ago

I'm trying to update everything so that when the new version of avrdude (7.1) comes out we are ready and can build a version that is production ready and that we can start to distribute to our users

I'm very excited about the Avrdude 7.1 release. It's packed with new and awesome features and functionality, and the fact that Arduino is ready to distribute a 7.1 so it can be bundled with 3rd party Arduino cores is very cool!

mcuee commented 1 year ago

Hi @mcuee, I just managed to release a test version of avrdude with the new container, I know, It took me a bit of time... 😓 It's built with crossbuild v0.2.2:

  • remove libusb-win32 dep in favor of libusb-compat
  • bump libncurses from 5.9 to 6.3 The base commit is avrdudes/avrdude@4c92030

Thanks a lot. The container is very useful. I am able to use the container to build avrdude git main without issues.

I'm trying to update everything so that when the new version of avrdude (7.1) comes out we are ready and can build a version that is production ready and that we can start to distribute to our users

That will be wonderful.

kristofmulier commented 1 year ago

Hi @umbynos , I don't have the time right now to test. But feel free to close the issue if you believe it is solved.

So, if I understand correctly, avrdude 7.1 will be released soon, and the libusb-problem I reported above will be fixed. That's great news.

@umbynos , @mariusgreuel and @mcuee , why is it necessary to have a separate AVRDUDE build for the Arduino boards? It's somewhat confusing to see two sources where I can collect my AVRDUDE binaries:

So I never really know which one to choose :-)

umbynos commented 1 year ago

Should be solved in https://github.com/arduino/avrdude-packing/releases/tag/7.1-arduino.1

kpishere commented 1 year ago

Tried the *alt.zip file on MacOS 10.15.7 and got this error.

avrdude usb_open_device() OS error: cannot claim interface, error code -13, Permission denied

Same error with other one too, used sudo also.

kpishere commented 1 year ago

@umbynos This isn't looking resolved to me. I tried 7.1 and got the following result.

$ sudo ~/Downloads/avrdude_macOS_64bit/bin/avrdude  -c pickit2 -p m2561 -v -V -t
Password:

avrdude: Version 7.1-arduino.1
         Copyright the AVRDUDE authors;
         see https://github.com/avrdudes/avrdude/blob/main/AUTHORS

         System wide configuration file is /Volumes/Scratch/Download/avrdude_macOS_64bit/etc/avrdude.conf
         User configuration file is /Users/user/.avrduderc
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : usb
         Using Programmer              : pickit2
device 0x7ff35ad05ae0 seemed to open OK
avrdude usb_open_device() OS error: cannot claim interface, error code -13, Permission denied
You may need to run avrdude as root or set up correct usb port permissions.         AVR Part                      : ATmega2561
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PA0
         RESET disposition             : possible i/o
         RETRY pulse                   : SCK
         Serial program mode           : yes
         Parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                           Block Poll               Page                       Polled
           Memory Type Alias    Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- -------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom                 65    10     8    0 no       4096    8      0  9000  9000 0x00 0x00
           flash                  65    10   256    0 yes    262144  256   1024  4500  4500 0x00 0x00
           lfuse                   0     0     0    0 no          1    1      0  9000  9000 0x00 0x00
           hfuse                   0     0     0    0 no          1    1      0  9000  9000 0x00 0x00
           efuse                   0     0     0    0 no          1    1      0  9000  9000 0x00 0x00
           lock                    0     0     0    0 no          1    1      0  9000  9000 0x00 0x00
           signature               0     0     0    0 no          3    1      0     0     0 0x00 0x00
           calibration             0     0     0    0 no          1    1      0     0     0 0x00 0x00

         Programmer Type : pickit2
         Description     : MicroChip's PICkit2 Programmer

avrdude pickit2_initialize() error: pickit2_write_report failed (ec -2). No such file or directory
avrdude main() error: initialization failed, rc=-1
        - double check the connections and try again
        - use -B to set lower ISP clock frequency, e.g. -B 125kHz
        - use -F to override this check

avrdude done.  Thank you.
mcuee commented 1 year ago

As of now, PICkit 2 is not supported under macOS. It has not been ported to HIDAPI yet.

Reference:

mcuee commented 1 year ago

@umbynos , @mariusgreuel and @mcuee , why is it necessary to have a separate AVRDUDE build for the Arduino boards? It's somewhat confusing to see two sources where I can collect my AVRDUDE binaries:

So I never really know which one to choose :-)

I have answered in other place.

Just to mention here as well.

As of now, avrdude project only provides official Windows binary (x86, x64 and ARM64). The expectation is that you get from the distros (eg: Linux distros, or Homebrew/Macports under macOS).

If you need cross-platform support, please use the binary release from Arduino avrdude-packing project.

kristofmulier commented 1 year ago

As of now, avrdude project only provides official Windows binary (x86, x64 and ARM64). The expectation is that you get from the distros (eg: Linux distros, or Homebrew/Macports under macOS).

That's a pity. Sometimes you don't want to "get it from the distros". Our Embeetle IDE is the typical use case: we just want to add a working avrdude-binary into our software. Our software doesn't need to be installed. You just download and unzip it - that's all. For this workflow, there's no point in "getting it from the distros".

If you need cross-platform support, please use the binary release from Arduino avrdude-packing project.

Thank God this exists. Otherwise, we'd be unable to put avrdude into our software.