avrdudes / avrdude

AVRDUDE is a utility to program AVR microcontrollers
GNU General Public License v2.0
734 stars 136 forks source link

To support NetBSD and OpenBSD #1177

Closed mcuee closed 1 year ago

mcuee commented 1 year ago

As of now NetBSD/pkgsrc only support avrdude 6.4. https://pkgsrc.se/cross/avrdude

As of now, OpenBSD only comes with avrdude 6.3. https://openports.se/devel/avrdude

It will be good to support avrdude 7.0 release and git main for NetBSD and OpenBSD.

This may not be high priority though.

dl8dtl commented 1 year ago

I think it's more important to have recent releases there rather than git main. It's a matter of someone taking over the job at NetBSD (and OpenBSD).

mcuee commented 1 year ago

Good point. Edit issue text to say avrdude 7.0 release and git main.

mcuee commented 1 year ago

I actally tried git main and it chocked at finding libusb.h in avrftdi_private.h.

mcuee commented 1 year ago

It's a matter of someone taking over the job at NetBSD (and OpenBSD).

Indeed. For example, for libusb project, OpenBSD is only up to 1.0.23 release whereas NetBSD/pkgsrc is at the latest 1.0.26 release.

However, for libusb project, I will at least do some sanity check before the release for NetBSD/OpenBSD. For example, the following is for libusb-1.0.26 (I did the same thing for 1.0.25).

dl8dtl commented 1 year ago

I actally tried git main and it chocked at finding libusb.h in avrftdi_private.h.

I guess they differ in that respect from FreeBSD. FreeBSD provides its own libusb implementation that covers the 0.1 and 1.0 APIs (plus one they call 2.0).

What do NetBSD and OpenBSD use? Something that ships as add-on package, or did they reuse the FreeBSD implementation?

mcuee commented 1 year ago

OpenBSD and NetBSD kind of use upstream libusb with some patches.

OpenBSD's Martin Pieuchot contributed the OpenBSD backend back in 2011 (commit was on 30-Jan-2012). https://github.com/libusb/libusb/commits?author=mpieuchot

Then I found that it also supports NetBSD and Pete added the commit to add NetBSD support here. https://github.com/libusb/libusb/commit/968df1244673e1e2b0e1acfffa9375d092d2c26d

We had to split NetBSD backend from OpenBSD backend here since no one from NetBSD jumped in to match further development from Martin. https://github.com/libusb/libusb/commit/eb4e7bee4488cefcf13929d9c4fe784650fef7c7

There is a ticket to improve NetBSD code from the OpenBSD code.

OpenBSD libusb is now at 1.0.23 release but with some patches. http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/devel/libusb1/

NetBSD/pkgsrc is already at the latest 1.0.26 release. It seems to me they do not add any patches. http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/devel/libusb1/

mcuee commented 1 year ago

Just tried under NetBSD and I have no problems build avrdude git main under NetBSD once I use the right build command.

cmake -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/pkg/lib
 -D CMAKE_BUILD_TYPE=RelWithDebInfo -B build_netbsd

cmake --build build_netbsd

sudo cmake --build build_netbsd --target install
mcuee commented 1 year ago

Under OpenBSD it has some compiling issues.

cmake -D CMAKE_C_FLAGS=-I/usr/local/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/local/lib
 -D CMAKE_BUILD_TYPE=RelWithDebInfo -B build_openbsd

cmake --build build_netbsd

Compiler errors. Sorry the clip board is not working so it is not easy for me to post the error messages. 1) fileio.c -- EM_AVR not defined 2) term.c -- incomplete type struct timeval 3) term.c -- undeclared fd_set and fds

dl8dtl commented 1 year ago

The first error means OpenBSD never updated their ELF header files.

The respective VCS log entry in FreeBSD dates back to 2008:

commit 926297fefb4c31902f11714276694d217f8e82c0
Author: Kai Wang <kaiw@FreeBSD.org>
Date:   Fri Dec 19 09:32:31 2008 +0000

    Update elf_common.h with new EM_ types found in the latest ELF gABI.

    This commit is slightly different from the original patch in the PR:
    1. EM_ALPHA keeps the old value for compatibility reason.
    2. Non-standard SHT_NUM is not added.
    3. Style.

    PR:                     kern/118540
    Submitted by:           "Pedro F. Giffuni" <giffunip[at]tutopia.com>

Well, we could hack around that with some #ifndef magic, but I'd prefer they adapt to the real world theirselves.

2 and 3 means they appear to no longer support the historical select(2) system interface. We could change the code to use poll(2) instead. If you want, you can open me a ticket for that, it's not going to be a major piece of work (only affects a few lines of code).

mcuee commented 1 year ago

I have created #1179.

I can see that term.c uses select.

What about ser_posix.c?

mcuee commented 1 year ago

The first error means OpenBSD never updated their ELF header files. The respective VCS log entry in FreeBSD dates back to 2008: Well, we could hack around that with some #ifndef magic, but I'd prefer they adapt to the real world theirselves.

The patch from OpenBSD is here. Looks like we have to use #ifndef magic. http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/devel/avrdude/patches/patch-fileio_c?rev=1.2&content-type=text/x-cvsweb-markup

Define EM_AVR, since it isn't included in our sys/exec_elf.h.

Index: fileio.c
--- fileio.c.orig
+++ fileio.c
@@ -34,6 +34,9 @@
 #elif defined(HAVE_LIBELF_LIBELF_H)
 #include <libelf/libelf.h>
 #endif
+#ifndef EM_AVR
+#define EM_AVR 83
+#endif
 #define EM_AVR32 0x18ad         /* inofficial */
 #endif
mcuee commented 1 year ago

@dl8dtl There is also a patch for stk500v2.c, not so sure why they need the patch. http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/devel/avrdude/patches/patch-stk500v2_c?rev=1.7&content-type=text/x-cvsweb-markup&hideattic=1

dl8dtl commented 1 year ago

Looks like we have to use #ifndef magic.

Too bad they still don't include that after 14 years … but we could do that.

dl8dtl commented 1 year ago

There is also a patch for stk500v2.c, not so sure why they need the patch.

Might make sense to include that one as well. It ensures the pointer don't dangle with old values. Not really an issue for the current CLI (because after a teardown, it will exit anway), but good idea for the library context.

dl8dtl commented 1 year ago

I have created #1179.

I can see that term.c uses select.

What about ser_posix.c?

The same.

Interesting that one compiles under OpenBSD, the other one doesn't. But then, neither includes <sys/select.h> which is the documented requirement.

(Even our Windows implementation uses select().)

mcuee commented 1 year ago

(Even our Windows implementation uses select().)

Indeed Windows provides the support from winsock2.h. https://learn.microsoft.com/en-us/windows/win32/api/winsock2/

mcuee commented 1 year ago

I have created #1179. I can see that term.c uses select. What about ser_posix.c?

The same.

Interesting that one compiles under OpenBSD, the other one doesn't. But then, neither includes <sys/select.h> which is the documented requirement.

@dl8dtl

Great, adding the <sys/select.h> sorted out the issue for OpenBSD.

I will try to create a PR for OpenBSD support.

mcuee commented 1 year ago

CLI mode seems to work fine under OpenBSD.

Screenshot 2022-11-16 081153

Terminal mode seems to work as well -- need to use the tip from @MCUdude to avoid timeout as this is inside a VM. Screenshot 2022-11-16 081706

dl8dtl commented 1 year ago

I will try to create a PR for OpenBSD support.

No, we are supposed to add this header to our sources that want to use select().

mcuee commented 1 year ago

I will try to create a PR for OpenBSD support.

No, we are supposed to add this header to our sources that want to use select().

@dl8dtl Yes I agree, we need to add <sys/select.h> header file for non-Win32 platform, for both term.c and ser_posix.c. This is not really for OpenBSD per se.

For fileio.c we will have to adopt the patch from OpenBSD as they are still using the very old version.

I am pretty bad at git, could you please create the PR? Thanks.

dl8dtl commented 1 year ago

https://github.com/avrdudes/avrdude/pull/1181

mcuee commented 1 year ago

1181

Thanks. This will fix fileio.c.

What about the <sys/select.h> includ file changes to term.c and ser_posix.c?

One more change could be build.sh to be able to build under NetBSD/OpenBSD out of box.

dl8dtl commented 1 year ago

I updated the PR to handle <sys/select.h>. I don't think it makes sense to have a separate PR just for that.

Regarding build.sh, the question is which -I and -L flags would be required for them.

mcuee commented 1 year ago

I updated the PR to handle <sys/select.h>. I don't think it makes sense to have a separate PR just for that.

I agree.

Regarding build.sh, the question is which -I and -L flags would be required for them.

OpenBSD -- same as FreeBSD.

build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/local/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/local/lib"

NetBSD (pkgsrc) -- using the following.

build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/pkg/lib"
dl8dtl commented 1 year ago

OK, added as well.

mcuee commented 1 year ago

OK, added as well.

Thanks. Now OpenBSD will work out of the box.

For NetBSD, the build works out of the box but the avrdude binary is not be able to find the libraries. Some further tuning of build.sh is required.

1) Adding -D CMAKE_INSTALL_PREFIX:PATH=/usr/pkg helps the installation 2) But the above is still not good enough.

Screenshot 2022-11-18 180245

mcuee commented 1 year ago

@dl8dtl Still I think you can merget #1181 as it is so that people who care about NetBSD will be able to jump in and contribute the change. Or you may want to remove the change to build.sh for NetBSD for now.

mcuee commented 1 year ago

PR #1181 has been merged. Openbsd should now work out of the box. The remaining issue is with NetBSD. As mentioned above, build.sh is not totally correct for NetBSD. Hopefully someone can help to fix the build issue under NetBSD.

mcuee commented 1 year ago

@dl8dtl

I have figured out how to build avrdude under NetBSD properly. We just need to add -R option for the linker.

cmake -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS="-R -L/usr/pkg/lib" -D CMAKE_BUILD_TYPE=RelWithDebInfo -B build_netbsd
cmake --build build_netbsd

But I have not figured how to get my Arduino Uno USB to Serial port to appear under NetBSD VM yet. Looks like the new VirtualBox version breaks USB support for NetBSD.

Ref: https://github.com/ocaml/opam/issues/3482#issuecomment-408066622

Screenshot 2022-12-03 183034

dl8dtl commented 1 year ago

We just need to add -R option for the linker.

That surprises me quite a bit. It records the LD_RUNPATH attribute. However, on a system using pkgsrc, I'd expect that /usr/pkg/lib is already handled by ldconfig properly – at least, that's the way all other operating systems usually handle this.

mcuee commented 1 year ago

We just need to add -R option for the linker.

That surprises me quite a bit. It records the LD_RUNPATH attribute. However, on a system using pkgsrc, I'd expect that /usr/pkg/lib is already handled by ldconfig properly – at least, that's the way all other operating systems usually handle this.

You are right. I checked on a different NetBSD installation under PVE vitual platform and the above -R solutiion does not seem to work out there.

mcuee commented 1 year ago

Failed run log:

localhost$ uname -a
NetBSD localhost 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 2022  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64

localhost$ cmake -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS="-R -L/usr/pkg/lib" -D CMAKE_BUILD_TYPE=RelWithDebInfo -B build_netbsd
cmake --build build_netbsd-- The C compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/pkg/bin/git (found version "2.37.1")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Found BISON: /usr/pkg/bin/bison (found version "3.8.2")
-- Looking for libelf.h
-- Looking for libelf.h - found
-- Looking for libelf/libelf.h
-- Looking for libelf/libelf.h - found
-- Looking for usb.h
-- Looking for usb.h - found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - not found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Configuration summary:
-- ----------------------
-- DO HAVE    libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DON'T HAVE libhidapi
-- DO HAVE    libftdi (but prefer to use libftdi1)
-- DO HAVE    libftdi1
-- DO HAVE    libreadline
-- DISABLED   doc
-- DISABLED   parport
-- DISABLED   linuxgpio
-- DISABLED   linuxspi
-- ----------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mcuee/avrdude/build_netbsd

localhost$ cmake --build build_netbsd
[  1%] Generating avrdude.conf
[  1%] Built target conf
[  3%] [BISON][Parser] Building parser with bison 3.8.2
[  4%] [FLEX][Parser] Building scanner with flex 2.6.4
[  6%] Building C object src/CMakeFiles/libavrdude.dir/arduino.c.o
[  7%] Building C object src/CMakeFiles/libavrdude.dir/avr.c.o
[  9%] Building C object src/CMakeFiles/libavrdude.dir/avr910.c.o
[ 10%] Building C object src/CMakeFiles/libavrdude.dir/avrcache.c.o
[ 12%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi.c.o
[ 13%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi_tpi.c.o
[ 15%] Building C object src/CMakeFiles/libavrdude.dir/avrpart.c.o
[ 16%] Building C object src/CMakeFiles/libavrdude.dir/bitbang.c.o
[ 18%] Building C object src/CMakeFiles/libavrdude.dir/buspirate.c.o
[ 19%] Building C object src/CMakeFiles/libavrdude.dir/butterfly.c.o
[ 21%] Building C object src/CMakeFiles/libavrdude.dir/config.c.o
[ 22%] Building C object src/CMakeFiles/libavrdude.dir/confwin.c.o
[ 24%] Building C object src/CMakeFiles/libavrdude.dir/crc16.c.o
[ 25%] Building C object src/CMakeFiles/libavrdude.dir/dfu.c.o
[ 27%] Building C object src/CMakeFiles/libavrdude.dir/fileio.c.o
[ 28%] Building C object src/CMakeFiles/libavrdude.dir/flip1.c.o
[ 30%] Building C object src/CMakeFiles/libavrdude.dir/flip2.c.o
[ 31%] Building C object src/CMakeFiles/libavrdude.dir/ft245r.c.o
[ 33%] Building C object src/CMakeFiles/libavrdude.dir/jtagmkI.c.o
[ 34%] Building C object src/CMakeFiles/libavrdude.dir/jtagmkII.c.o
[ 36%] Building C object src/CMakeFiles/libavrdude.dir/jtag3.c.o
[ 37%] Building C object src/CMakeFiles/libavrdude.dir/linuxgpio.c.o
[ 39%] Building C object src/CMakeFiles/libavrdude.dir/linuxspi.c.o
[ 40%] Building C object src/CMakeFiles/libavrdude.dir/lists.c.o
[ 42%] Building C object src/CMakeFiles/libavrdude.dir/micronucleus.c.o
[ 43%] Building C object src/CMakeFiles/libavrdude.dir/par.c.o
[ 45%] Building C object src/CMakeFiles/libavrdude.dir/pgm.c.o
[ 46%] Building C object src/CMakeFiles/libavrdude.dir/pgm_type.c.o
[ 48%] Building C object src/CMakeFiles/libavrdude.dir/pickit2.c.o
[ 50%] Building C object src/CMakeFiles/libavrdude.dir/pindefs.c.o
[ 51%] Building C object src/CMakeFiles/libavrdude.dir/ppi.c.o
[ 53%] Building C object src/CMakeFiles/libavrdude.dir/ppiwin.c.o
[ 54%] Building C object src/CMakeFiles/libavrdude.dir/serbb_posix.c.o
[ 56%] Building C object src/CMakeFiles/libavrdude.dir/serbb_win32.c.o
[ 57%] Building C object src/CMakeFiles/libavrdude.dir/ser_avrdoper.c.o
[ 59%] Building C object src/CMakeFiles/libavrdude.dir/ser_posix.c.o
[ 60%] Building C object src/CMakeFiles/libavrdude.dir/ser_win32.c.o
[ 62%] Building C object src/CMakeFiles/libavrdude.dir/serialupdi.c.o
[ 63%] Building C object src/CMakeFiles/libavrdude.dir/stk500.c.o
[ 65%] Building C object src/CMakeFiles/libavrdude.dir/stk500v2.c.o
[ 66%] Building C object src/CMakeFiles/libavrdude.dir/stk500generic.c.o
[ 68%] Building C object src/CMakeFiles/libavrdude.dir/teensy.c.o
[ 69%] Building C object src/CMakeFiles/libavrdude.dir/updi_link.c.o
[ 71%] Building C object src/CMakeFiles/libavrdude.dir/updi_nvm.c.o
[ 72%] Building C object src/CMakeFiles/libavrdude.dir/updi_readwrite.c.o
[ 74%] Building C object src/CMakeFiles/libavrdude.dir/updi_state.c.o
[ 75%] Building C object src/CMakeFiles/libavrdude.dir/urclock.c.o
[ 77%] Building C object src/CMakeFiles/libavrdude.dir/usbasp.c.o
[ 78%] Building C object src/CMakeFiles/libavrdude.dir/usb_hidapi.c.o
[ 80%] Building C object src/CMakeFiles/libavrdude.dir/usb_libusb.c.o
[ 81%] Building C object src/CMakeFiles/libavrdude.dir/usbtiny.c.o
[ 83%] Building C object src/CMakeFiles/libavrdude.dir/update.c.o
[ 84%] Building C object src/CMakeFiles/libavrdude.dir/wiring.c.o
[ 86%] Building C object src/CMakeFiles/libavrdude.dir/xbee.c.o
[ 87%] Building C object src/CMakeFiles/libavrdude.dir/__/lexer.c.o
[ 89%] Building C object src/CMakeFiles/libavrdude.dir/__/config_gram.c.o
[ 90%] Linking C static library libavrdude.a
[ 90%] Built target libavrdude
[ 92%] Building C object src/CMakeFiles/avrdude.dir/main.c.o
[ 93%] Building C object src/CMakeFiles/avrdude.dir/term.c.o
[ 95%] Building C object src/CMakeFiles/avrdude.dir/avrintel.c.o
[ 96%] Building C object src/CMakeFiles/avrdude.dir/developer_opts.c.o
[ 98%] Building C object src/CMakeFiles/avrdude.dir/whereami.c.o
[100%] Linking C executable avrdude
[100%] Built target avrdude
localhost$ ldd ./build_netbsd/src/avrdude
./build_netbsd/src/avrdude:
        -lm.0 => /usr/lib/libm.so.0
        -lc.12 => /usr/lib/libc.so.12
        -lelf.0 => not found
        -lusb-0.1.4 => not found
        -lusb-1.0.0 => not found
        -lhid.0 => not found
        -lftdi.1 => not found
        -lftdi1.2 => not found
        -lreadline.8 => not found
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1

Verbose linking step:

[100%] Linking C executable avrdude
cd /home/mcuee/avrdude/build_netbsd/src && /usr/pkg/bin/cmake -E cmake_link_script CMakeFiles/avrdude.dir/link.txt --verbose=1
/usr/bin/cc -I/usr/pkg/include -O2 -g -DNDEBUG -R -L/usr/pkg/lib CMakeFiles/avrdude.dir/main.c.o CMakeFiles/avrdude.dir/term.c.o CMakeFiles/avrdude.dir/avrintel.c.o CMakeFiles/avrdude.dir/developer_opts.c.o CMakeFiles/avrdude.dir/whereami.c.o -o avrdude  libavrdude.a -lm /usr/pkg/lib/libelf.so /usr/pkg/lib/libusb.so /usr/pkg/lib/libusb-1.0.so /usr/pkg/lib/libhid.so /usr/pkg/lib/libftdi.so /usr/pkg/lib/libftdi1.so /usr/pkg/lib/libreadline.so
[100%] Built target avrdude
mcuee commented 1 year ago

@dl8dtl

I think I find the reason, LD_LIBRARY_PATH is not set and that is the reason.

localhost$ pwd
/home/mcuee/avrdude/build_netbsd/src

localhost$ echo $LD_LIBRARY_PATH

localhost$ LD_LIBRARY_PATH=/usr/pkg/lib ./avrdude
Usage: avrdude [options]
Options:
  -p <partno>                Specify AVR device
  -b <baudrate>              Override RS-232 baud rate
  -B <bitclock>              Specify bit clock period (us)
  -C <config-file>           Specify location of configuration file
  -c <programmer>            Specify programmer type
  -A                         Disable trailing-0xff removal from file and AVR read
  -D                         Disable auto erase for flash memory; implies -A
  -i <delay>                 ISP Clock Delay [in microseconds]
  -P <port>                  Specify connection port
  -F                         Override invalid signature or initialisation check
  -e                         Perform a chip erase
  -O                         Perform RC oscillator calibration (see AVR053)
  -U <memtype>:r|w|v:<filename>[:format]
                             Memory operation specification
                             Multiple -U options are allowed, each request
                             is performed in the order specified
  -n                         Do not write anything to the device
  -V                         Do not verify
  -t                         Enter terminal mode
  -E <exitspec>[,<exitspec>] List programmer exit specifications
  -x <extended_param>        Pass <extended_param> to programmer
  -v                         Verbose output; -v -v for more
  -q                         Quell progress output; -q -q for less
  -l logfile                 Use logfile rather than stderr for diagnostics
  -?                         Display this usage

avrdude version 7.0-20221129 (ea2d7ee), URL: <https://github.com/avrdudes/avrdude>
mcuee commented 1 year ago

Full build log using default build.sh. Tested to be okay under NetBSD 9.3 and 9.2.

localhost$ LD_LIBRARY_PATH=/usr/pkg/lib

localhost$ export  LD_LIBRARY_PATH

localhost$ echo $LD_LIBRARY_PATH
/usr/pkg/lib

localhost$ git clone https://github.com/avrdudes/avrdude
Cloning into 'avrdude'...
remote: Enumerating objects: 14691, done.
remote: Counting objects: 100% (597/597), done.
remote: Compressing objects: 100% (249/249), done.
remote: Total 14691 (delta 407), reused 514 (delta 344), pack-reused 14094
Receiving objects: 100% (14691/14691), 7.93 MiB | 4.10 MiB/s, done.
Resolving deltas: 100% (10286/10286), done.

localhost$ cd avrdude/

localhost$ ./build.sh
-- The C compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/pkg/bin/git (found version "2.37.1")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Found BISON: /usr/pkg/bin/bison (found version "3.8.2")
-- Looking for libelf.h
-- Looking for libelf.h - found
-- Looking for libelf/libelf.h
-- Looking for libelf/libelf.h - found
-- Looking for usb.h
-- Looking for usb.h - found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - not found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Configuration summary:
-- ----------------------
-- DO HAVE    libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DON'T HAVE libhidapi
-- DO HAVE    libftdi (but prefer to use libftdi1)
-- DO HAVE    libftdi1
-- DO HAVE    libreadline
-- DISABLED   doc
-- DISABLED   parport
-- DISABLED   linuxgpio
-- DISABLED   linuxspi
-- ----------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mcuee/avrdude/build_netbsd
[  1%] Generating avrdude.conf
[  1%] Built target conf
[  3%] [BISON][Parser] Building parser with bison 3.8.2
[  4%] [FLEX][Parser] Building scanner with flex 2.6.4
[  6%] Building C object src/CMakeFiles/libavrdude.dir/arduino.c.o
[  7%] Building C object src/CMakeFiles/libavrdude.dir/avr.c.o
[  9%] Building C object src/CMakeFiles/libavrdude.dir/avr910.c.o
[ 10%] Building C object src/CMakeFiles/libavrdude.dir/avrcache.c.o
[ 12%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi.c.o
[ 13%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi_tpi.c.o
[ 15%] Building C object src/CMakeFiles/libavrdude.dir/avrpart.c.o
[ 16%] Building C object src/CMakeFiles/libavrdude.dir/bitbang.c.o
[ 18%] Building C object src/CMakeFiles/libavrdude.dir/buspirate.c.o
[ 19%] Building C object src/CMakeFiles/libavrdude.dir/butterfly.c.o
[ 21%] Building C object src/CMakeFiles/libavrdude.dir/config.c.o
[ 22%] Building C object src/CMakeFiles/libavrdude.dir/confwin.c.o
[ 24%] Building C object src/CMakeFiles/libavrdude.dir/crc16.c.o
[ 25%] Building C object src/CMakeFiles/libavrdude.dir/dfu.c.o
[ 27%] Building C object src/CMakeFiles/libavrdude.dir/fileio.c.o
[ 28%] Building C object src/CMakeFiles/libavrdude.dir/flip1.c.o
[ 30%] Building C object src/CMakeFiles/libavrdude.dir/flip2.c.o
[ 31%] Building C object src/CMakeFiles/libavrdude.dir/ft245r.c.o
[ 33%] Building C object src/CMakeFiles/libavrdude.dir/jtagmkI.c.o
[ 34%] Building C object src/CMakeFiles/libavrdude.dir/jtagmkII.c.o
[ 36%] Building C object src/CMakeFiles/libavrdude.dir/jtag3.c.o
[ 37%] Building C object src/CMakeFiles/libavrdude.dir/linuxgpio.c.o
[ 39%] Building C object src/CMakeFiles/libavrdude.dir/linuxspi.c.o
[ 40%] Building C object src/CMakeFiles/libavrdude.dir/lists.c.o
[ 42%] Building C object src/CMakeFiles/libavrdude.dir/micronucleus.c.o
[ 43%] Building C object src/CMakeFiles/libavrdude.dir/par.c.o
[ 45%] Building C object src/CMakeFiles/libavrdude.dir/pgm.c.o
[ 46%] Building C object src/CMakeFiles/libavrdude.dir/pgm_type.c.o
[ 48%] Building C object src/CMakeFiles/libavrdude.dir/pickit2.c.o
[ 50%] Building C object src/CMakeFiles/libavrdude.dir/pindefs.c.o
[ 51%] Building C object src/CMakeFiles/libavrdude.dir/ppi.c.o
[ 53%] Building C object src/CMakeFiles/libavrdude.dir/ppiwin.c.o
[ 54%] Building C object src/CMakeFiles/libavrdude.dir/serbb_posix.c.o
[ 56%] Building C object src/CMakeFiles/libavrdude.dir/serbb_win32.c.o
[ 57%] Building C object src/CMakeFiles/libavrdude.dir/ser_avrdoper.c.o
[ 59%] Building C object src/CMakeFiles/libavrdude.dir/ser_posix.c.o
[ 60%] Building C object src/CMakeFiles/libavrdude.dir/ser_win32.c.o
[ 62%] Building C object src/CMakeFiles/libavrdude.dir/serialupdi.c.o
[ 63%] Building C object src/CMakeFiles/libavrdude.dir/stk500.c.o
[ 65%] Building C object src/CMakeFiles/libavrdude.dir/stk500v2.c.o
[ 66%] Building C object src/CMakeFiles/libavrdude.dir/stk500generic.c.o
[ 68%] Building C object src/CMakeFiles/libavrdude.dir/teensy.c.o
[ 69%] Building C object src/CMakeFiles/libavrdude.dir/updi_link.c.o
[ 71%] Building C object src/CMakeFiles/libavrdude.dir/updi_nvm.c.o
[ 72%] Building C object src/CMakeFiles/libavrdude.dir/updi_readwrite.c.o
[ 74%] Building C object src/CMakeFiles/libavrdude.dir/updi_state.c.o
[ 75%] Building C object src/CMakeFiles/libavrdude.dir/urclock.c.o
[ 77%] Building C object src/CMakeFiles/libavrdude.dir/usbasp.c.o
[ 78%] Building C object src/CMakeFiles/libavrdude.dir/usb_hidapi.c.o
[ 80%] Building C object src/CMakeFiles/libavrdude.dir/usb_libusb.c.o
[ 81%] Building C object src/CMakeFiles/libavrdude.dir/usbtiny.c.o
[ 83%] Building C object src/CMakeFiles/libavrdude.dir/update.c.o
[ 84%] Building C object src/CMakeFiles/libavrdude.dir/wiring.c.o
[ 86%] Building C object src/CMakeFiles/libavrdude.dir/xbee.c.o
[ 87%] Building C object src/CMakeFiles/libavrdude.dir/__/lexer.c.o
[ 89%] Building C object src/CMakeFiles/libavrdude.dir/__/config_gram.c.o
[ 90%] Linking C static library libavrdude.a
[ 90%] Built target libavrdude
[ 92%] Building C object src/CMakeFiles/avrdude.dir/main.c.o
[ 93%] Building C object src/CMakeFiles/avrdude.dir/term.c.o
[ 95%] Building C object src/CMakeFiles/avrdude.dir/avrintel.c.o
[ 96%] Building C object src/CMakeFiles/avrdude.dir/developer_opts.c.o
[ 98%] Building C object src/CMakeFiles/avrdude.dir/whereami.c.o
[100%] Linking C executable avrdude
[100%] Built target avrdude

Build succeeded.

Run

sudo cmake --build build_netbsd --target install

to install.

localhost$ ldd ./build_netbsd/src/avrdude
./build_netbsd/src/avrdude:
        -lm.0 => /usr/lib/libm.so.0
        -lc.12 => /usr/lib/libc.so.12
        -lelf.0 => /usr/pkg/lib/libelf.so.0
        -lusb-0.1.4 => /usr/pkg/lib/libusb-0.1.so.4
        -lusb-1.0.0 => /usr/pkg/lib/libusb-1.0.so.0
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lhid.0 => /usr/pkg/lib/libhid.so.0
        -lftdi.1 => /usr/pkg/lib/libftdi.so.1
        -lftdi1.2 => /usr/pkg/lib/libftdi1.so.2
        -lreadline.8 => /usr/pkg/lib/libreadline.so.8
        -lterminfo.1 => /usr/lib/libterminfo.so.1
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
mcuee commented 1 year ago

I can not attach USB devices to NetBSD 9.2 VM under Windows Virtual box. But I can attach COM port.

And avrdude seems to work fine. As it uses GNU Readline, -c urclock -t also works fine.

Screenshot 2022-12-04 151350

Screenshot 2022-12-04 152806

mcuee commented 1 year ago

I think I can close this issue now.

dl8dtl commented 1 year ago

LD_LIBRARY_PATH is not set

But again, on a system that uses pkgsrc, I'd expect that /usr/pkg/lib is pre-configured system-wide using ldconfig. That has the additional advantage that e.g. setuid binaries will work using libraries from that location (they do not honor LD_LIBRARY_PATH or any other user-provided location for security reasons).

I tried looking up their pkgsrc guide, but could not find any statement about it.

Well, it seems the NetBSD folks don't really like ldconfig and ld.so.conf, and prefer recording the library path in the binary:

https://www.netbsd.org/docs/elf.html

Consequently, your solution with adding -R to the linker options is the right thing to do.

mcuee commented 1 year ago

@dl8dtl

Great. Now we have a simple solution. I tested the following simple patch and it works fine. Please help to update build.sh accordingly. Thanks.

localhost$ git diff
diff --git a/build.sh b/build.sh
index d5685a0..bf9a9bc 100755
--- a/build.sh
+++ b/build.sh
@@ -61,7 +61,7 @@ case "${ostype}" in
        ;;

     netbsd)
-       build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/pkg/lib"
+       build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-R/usr/pkg/lib"
        ;;

     *bsd)
localhost$ ldd ./build_netbsd/src/avrdude
./build_netbsd/src/avrdude:
        -lm.0 => /usr/lib/libm.so.0
        -lc.12 => /usr/lib/libc.so.12
        -lelf.0 => /usr/pkg/lib/libelf.so.0
        -lusb-0.1.4 => /usr/pkg/lib/libusb-0.1.so.4
        -lusb-1.0.0 => /usr/pkg/lib/libusb-1.0.so.0
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lhid.0 => /usr/pkg/lib/libhid.so.0
        -lftdi.1 => /usr/pkg/lib/libftdi.so.1
        -lftdi1.2 => /usr/pkg/lib/libftdi1.so.2
        -lreadline.8 => /usr/pkg/lib/libreadline.so.8
        -lterminfo.1 => /usr/lib/libterminfo.so.1
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
mcuee commented 1 year ago

Full runlog.

localhost$ git clone https://github.com/avrdudes/avrdude avrdude_git
Cloning into 'avrdude_git'...
remote: Enumerating objects: 14691, done.
remote: Counting objects: 100% (597/597), done.
remote: Compressing objects: 100% (253/253), done.
remote: Total 14691 (delta 406), reused 511 (delta 340), pack-reused 14094
Receiving objects: 100% (14691/14691), 7.94 MiB | 5.04 MiB/s, done.
Resolving deltas: 100% (10286/10286), done.

localhost$ cd avrdude_git/

localhost$ cp build.sh build_org.sh

localhost$ nano build.sh

localhost$ git diff
diff --git a/build.sh b/build.sh
index d5685a0..69ad0ae 100755
--- a/build.sh
+++ b/build.sh
@@ -61,7 +61,7 @@ case "${ostype}" in
        ;;

     netbsd)
-       build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/pkg/lib"
+       build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-R/usr/pkg/lib -D CMAKE_INSTALL_PREFIX:PATH=/usr/pkg"
        ;;

     *bsd)

localhost$ ./build.sh
-- The C compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/pkg/bin/git (found version "2.37.1")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Found BISON: /usr/pkg/bin/bison (found version "3.8.2")
-- Looking for libelf.h
-- Looking for libelf.h - found
-- Looking for libelf/libelf.h
-- Looking for libelf/libelf.h - found
-- Looking for usb.h
-- Looking for usb.h - found
-- Looking for lusb0_usb.h
-- Looking for lusb0_usb.h - not found
-- Looking for libusb.h
-- Looking for libusb.h - not found
-- Looking for libusb-1.0/libusb.h
-- Looking for libusb-1.0/libusb.h - found
-- Looking for ftdi_tcioflush
-- Looking for ftdi_tcioflush - found
-- Configuration summary:
-- ----------------------
-- DO HAVE    libelf
-- DO HAVE    libusb
-- DO HAVE    libusb_1_0
-- DON'T HAVE libhidapi
-- DO HAVE    libftdi (but prefer to use libftdi1)
-- DO HAVE    libftdi1
-- DO HAVE    libreadline
-- DISABLED   doc
-- DISABLED   parport
-- DISABLED   linuxgpio
-- DISABLED   linuxspi
-- ----------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mcuee/avrdude_git/build_netbsd
[  1%] Generating avrdude.conf
[  1%] Built target conf
[  3%] [BISON][Parser] Building parser with bison 3.8.2
[  4%] [FLEX][Parser] Building scanner with flex 2.6.4
[  6%] Building C object src/CMakeFiles/libavrdude.dir/arduino.c.o
[  7%] Building C object src/CMakeFiles/libavrdude.dir/avr.c.o
[  9%] Building C object src/CMakeFiles/libavrdude.dir/avr910.c.o
[ 10%] Building C object src/CMakeFiles/libavrdude.dir/avrcache.c.o
[ 12%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi.c.o
[ 13%] Building C object src/CMakeFiles/libavrdude.dir/avrftdi_tpi.c.o
[ 15%] Building C object src/CMakeFiles/libavrdude.dir/avrpart.c.o
[ 16%] Building C object src/CMakeFiles/libavrdude.dir/bitbang.c.o
[ 18%] Building C object src/CMakeFiles/libavrdude.dir/buspirate.c.o
[ 19%] Building C object src/CMakeFiles/libavrdude.dir/butterfly.c.o
[ 21%] Building C object src/CMakeFiles/libavrdude.dir/config.c.o
[ 22%] Building C object src/CMakeFiles/libavrdude.dir/confwin.c.o
[ 24%] Building C object src/CMakeFiles/libavrdude.dir/crc16.c.o
[ 25%] Building C object src/CMakeFiles/libavrdude.dir/dfu.c.o
[ 27%] Building C object src/CMakeFiles/libavrdude.dir/fileio.c.o
[ 28%] Building C object src/CMakeFiles/libavrdude.dir/flip1.c.o
[ 30%] Building C object src/CMakeFiles/libavrdude.dir/flip2.c.o
[ 31%] Building C object src/CMakeFiles/libavrdude.dir/ft245r.c.o
[ 33%] Building C object src/CMakeFiles/libavrdude.dir/jtagmkI.c.o
[ 34%] Building C object src/CMakeFiles/libavrdude.dir/jtagmkII.c.o
[ 36%] Building C object src/CMakeFiles/libavrdude.dir/jtag3.c.o
[ 37%] Building C object src/CMakeFiles/libavrdude.dir/linuxgpio.c.o
[ 39%] Building C object src/CMakeFiles/libavrdude.dir/linuxspi.c.o
[ 40%] Building C object src/CMakeFiles/libavrdude.dir/lists.c.o
[ 42%] Building C object src/CMakeFiles/libavrdude.dir/micronucleus.c.o
[ 43%] Building C object src/CMakeFiles/libavrdude.dir/par.c.o
[ 45%] Building C object src/CMakeFiles/libavrdude.dir/pgm.c.o
[ 46%] Building C object src/CMakeFiles/libavrdude.dir/pgm_type.c.o
[ 48%] Building C object src/CMakeFiles/libavrdude.dir/pickit2.c.o
[ 50%] Building C object src/CMakeFiles/libavrdude.dir/pindefs.c.o
[ 51%] Building C object src/CMakeFiles/libavrdude.dir/ppi.c.o
[ 53%] Building C object src/CMakeFiles/libavrdude.dir/ppiwin.c.o
[ 54%] Building C object src/CMakeFiles/libavrdude.dir/serbb_posix.c.o
[ 56%] Building C object src/CMakeFiles/libavrdude.dir/serbb_win32.c.o
[ 57%] Building C object src/CMakeFiles/libavrdude.dir/ser_avrdoper.c.o
[ 59%] Building C object src/CMakeFiles/libavrdude.dir/ser_posix.c.o
[ 60%] Building C object src/CMakeFiles/libavrdude.dir/ser_win32.c.o
[ 62%] Building C object src/CMakeFiles/libavrdude.dir/serialupdi.c.o
[ 63%] Building C object src/CMakeFiles/libavrdude.dir/stk500.c.o
[ 65%] Building C object src/CMakeFiles/libavrdude.dir/stk500v2.c.o
[ 66%] Building C object src/CMakeFiles/libavrdude.dir/stk500generic.c.o
[ 68%] Building C object src/CMakeFiles/libavrdude.dir/teensy.c.o
[ 69%] Building C object src/CMakeFiles/libavrdude.dir/updi_link.c.o
[ 71%] Building C object src/CMakeFiles/libavrdude.dir/updi_nvm.c.o
[ 72%] Building C object src/CMakeFiles/libavrdude.dir/updi_readwrite.c.o
[ 74%] Building C object src/CMakeFiles/libavrdude.dir/updi_state.c.o
[ 75%] Building C object src/CMakeFiles/libavrdude.dir/urclock.c.o
[ 77%] Building C object src/CMakeFiles/libavrdude.dir/usbasp.c.o
[ 78%] Building C object src/CMakeFiles/libavrdude.dir/usb_hidapi.c.o
[ 80%] Building C object src/CMakeFiles/libavrdude.dir/usb_libusb.c.o
[ 81%] Building C object src/CMakeFiles/libavrdude.dir/usbtiny.c.o
[ 83%] Building C object src/CMakeFiles/libavrdude.dir/update.c.o
[ 84%] Building C object src/CMakeFiles/libavrdude.dir/wiring.c.o
[ 86%] Building C object src/CMakeFiles/libavrdude.dir/xbee.c.o
[ 87%] Building C object src/CMakeFiles/libavrdude.dir/__/lexer.c.o
[ 89%] Building C object src/CMakeFiles/libavrdude.dir/__/config_gram.c.o
[ 90%] Linking C static library libavrdude.a
[ 90%] Built target libavrdude
[ 92%] Building C object src/CMakeFiles/avrdude.dir/main.c.o
[ 93%] Building C object src/CMakeFiles/avrdude.dir/term.c.o
[ 95%] Building C object src/CMakeFiles/avrdude.dir/avrintel.c.o
[ 96%] Building C object src/CMakeFiles/avrdude.dir/developer_opts.c.o
[ 98%] Building C object src/CMakeFiles/avrdude.dir/whereami.c.o
[100%] Linking C executable avrdude
[100%] Built target avrdude

Build succeeded.

Run

sudo cmake --build build_netbsd --target install

to install.

localhost$ sudo cmake --build build_netbsd --target install
Password:
[  1%] Built target conf
Consolidate compiler generated dependencies of target libavrdude
[ 90%] Built target libavrdude
Consolidate compiler generated dependencies of target avrdude
[100%] Built target avrdude
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Installing: /usr/pkg/bin/avrdude
-- Set runtime path of "/usr/pkg/bin/avrdude" to ""
-- Installing: /usr/pkg/lib/libavrdude.a
-- Installing: /usr/pkg/include/libavrdude.h
-- Installing: /usr/pkg/etc/avrdude.conf
-- Installing: /usr/pkg/man/man1/avrdude.1

localhost$ ldd /usr/pkg/bin/avrdude
/usr/pkg/bin/avrdude:
        -lm.0 => /usr/lib/libm.so.0
        -lc.12 => /usr/lib/libc.so.12
        -lelf.0 => /usr/pkg/lib/libelf.so.0
        -lusb-0.1.4 => /usr/pkg/lib/libusb-0.1.so.4
        -lusb-1.0.0 => /usr/pkg/lib/libusb-1.0.so.0
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lhid.0 => /usr/pkg/lib/libhid.so.0
        -lftdi.1 => /usr/pkg/lib/libftdi.so.1
        -lftdi1.2 => /usr/pkg/lib/libftdi1.so.2
        -lreadline.8 => /usr/pkg/lib/libreadline.so.8
        -lterminfo.1 => /usr/lib/libterminfo.so.1
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1

localhost$ avrdude
Usage: avrdude [options]
Options:
  -p <partno>                Specify AVR device
  -b <baudrate>              Override RS-232 baud rate
  -B <bitclock>              Specify bit clock period (us)
  -C <config-file>           Specify location of configuration file
  -c <programmer>            Specify programmer type
  -A                         Disable trailing-0xff removal from file and AVR read
  -D                         Disable auto erase for flash memory; implies -A
  -i <delay>                 ISP Clock Delay [in microseconds]
  -P <port>                  Specify connection port
  -F                         Override invalid signature or initialisation check
  -e                         Perform a chip erase
  -O                         Perform RC oscillator calibration (see AVR053)
  -U <memtype>:r|w|v:<filename>[:format]
                             Memory operation specification
                             Multiple -U options are allowed, each request
                             is performed in the order specified
  -n                         Do not write anything to the device
  -V                         Do not verify
  -t                         Enter terminal mode
  -E <exitspec>[,<exitspec>] List programmer exit specifications
  -x <extended_param>        Pass <extended_param> to programmer
  -v                         Verbose output; -v -v for more
  -q                         Quell progress output; -q -q for less
  -l logfile                 Use logfile rather than stderr for diagnostics
  -?                         Display this usage

avrdude version 7.0-20221129 (ea2d7ee), URL: <https://github.com/avrdudes/avrdude>
mcuee commented 1 year ago

@dl8dtl

Please help to update build.sh accordingly for NetBSD when you have the time to work on it. Thanks.

The installation prefix is needed to get the installation working correctly.

localhost$ git diff
diff --git a/build.sh b/build.sh
index d5685a0..69ad0ae 100755
--- a/build.sh
+++ b/build.sh
@@ -61,7 +61,7 @@ case "${ostype}" in
        ;;

     netbsd)
-       build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-L/usr/pkg/lib"
+       build_flags="${build_flags} -D CMAKE_C_FLAGS=-I/usr/pkg/include -D CMAKE_EXE_LINKER_FLAGS=-R/usr/pkg/lib -D CMAKE_INSTALL_PREFIX:PATH=/usr/pkg"
        ;;

     *bsd)
dl8dtl commented 1 year ago

Reopening to handle build.sh

mcuee commented 1 year ago

@dl8dtl

As of now. build.sh is not correct for HomeBrew either when it comes to installation path. I am not sure how to handle that. It is supposed to install to the Cellar (with the version) and then the user needs to use brew link. Last time there was a brew diy helper but I think that is deprecated.

I think we can ignore that for now.

The easy way is to use the following commands if one wants to follow git main.

brew install avrdude --head (one time)
brew reinstall avrude (whenever the users want to upgrade to latest git main)

If you want to look at the details, here is the Homebrew avrdude formula. https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/avrdude.rb

I tend to think -DCMAKE_SHARED_LINKER_FLAGS=-Wl,-undefined,dynamic_lookup is no longer required.

mcuee commented 1 year ago

I have installed a new NetBSD 9.3 Virtualbox VM under Windows and now USB works fine.

Screenshot 2022-12-09 121143

mcuee commented 1 year ago

@dl8dtl

The basic fix to NetBSD build.sh is good to have before 7.1 release. Thanks.

mcuee commented 1 year ago

I have merged #1241 and this issue can be closed.