Better documentation for compiling optiboot

[mcuee]

I think Wiki is a good start as well as README.TXT. https://github.com/Optiboot/optiboot/wiki/CompilingOptiboot https://github.com/Optiboot/optiboot/blob/master/optiboot/bootloaders/optiboot/README.TXT

But there are a few questions. Toolchain -- is WinAVR really supported now? Is Atmel toolchain (avr-gcc 5.4 based and 7.3 based) supported?

I believe WinAVR is no longer supported. So it is better to remove it from the Wiki. I think Atmel toolchain links should be provided in the Wiki and they should work based on my testing.

https://www.microchip.com/en-us/tools-resources/develop/microchip-studio/gcc-compilers Atmel version 3.7.0 --> avr-gcc 7.3.0 Atmel version 3.6.2 --> avr-gcc 5.4.0

README.TXT -- which part is still relevant?

Build environment under Windows -- is CMD really supported? What about PowerShell and MSYS2 mingw?

I have no issued building under Linux and macOS. Windows is not a big problem but the building process needs to be better documented. I need to use git bash to get it work or use MSYS2.

Build environment under macOS -- CrossPacks is too old to work for newe version of macOS. It is better to only mention Arduino toolchains which works out of the box.

[mcuee]

Example under Windows using CMD or PowerShell.

c:\work\avr\avrdude_test\others\optiboot\optiboot\bootloaders\optiboot>make
avr-gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

process_begin: CreateProcess(NULL, bash baudcheck.tmp.sh, ...) failed.
make (e=2): The system cannot find the file specified.
make: [Makefile:684: baudcheck] Error 2 (ignored)
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
 #warning BAUD_RATE off by greater than 2%
  ^~~~~~~
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num' -Wl,-Tlink_optiboot.ld -Wl,--relax -nostartfiles -o optiboot_virboot8.elf optiboot.o
avr-size optiboot_virboot8.elf
   text    data     bss     dec     hex filename
    602       0       0     602     25a optiboot_virboot8.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_virboot8.elf optiboot_virboot8.hex
avr-objdump -h -S optiboot_virboot8.elf > optiboot_virboot8.lst
rm optiboot.o

c:\work\avr\avrdude_test\others\optiboot\optiboot\bootloaders\optiboot>make clean
Too many tokens found on command line.  First unexpected token was
"__temp_.elf".
*** Program aborted on error. ***
make: *** [Makefile:699: clean] Error 3

c:\work\avr\avrdude_test\others\optiboot\optiboot\bootloaders\optiboot>make clean
Too many tokens found on command line.  First unexpected token was
"__temp_.elf".
*** Program aborted on error. ***
make: *** [Makefile:699: clean] Error 3

[mcuee]

Under Windows I need to use git bash to get it working,

$ make clean
rm -rf *.o *.elf *.lst *.map *.sym *.lss *.eep *.srec *.bin *.hex *.tmp.sh

$ make
avr-gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

BAUD RATE CHECK: Desired: 115200, Real: 117647, UBRRL = 16, Difference=2.1%
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
 #warning BAUD_RATE off by greater than 2%
  ^~~~~~~
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num' -Wl,-Tlink_optiboot.ld -Wl,--relax -nostartfiles -o optiboot_virboot8.elf optiboot.o
avr-size optiboot_virboot8.elf
   text    data     bss     dec     hex filename
    602       0       0     602     25a optiboot_virboot8.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_virboot8.elf optiboot_virboot8.hex
avr-objdump -h -S optiboot_virboot8.elf > optiboot_virboot8.lst
rm optiboot.o

[mcuee]

MSYS2 works as well with its bundled toolchain.

$ make clean
rm -rf *.o *.elf *.lst *.map *.sym *.lss *.eep *.srec *.bin *.hex *.tmp.sh

$ make
avr-gcc.exe (GCC) 8.5.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

BAUD RATE CHECK: Desired: 115200, Real: 117647, UBRRL = 16, Difference=2.1%
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
 #warning BAUD_RATE off by greater than 2%
  ^~~~~~~
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num' -Wl,-Tlink_optiboot.ld -Wl,--relax -nostartfiles -o optiboot_virboot8.elf optiboot.o
avr-size optiboot_virboot8.elf
   text    data     bss     dec     hex filename
    602       0       0     602     25a optiboot_virboot8.elf
avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot_virboot8.elf optiboot_virboot8.hex
avr-objdump -h -S optiboot_virboot8.elf > optiboot_virboot8.lst
rm optiboot.o

$ avr-gcc -v
Using built-in specs.
Reading specs from c:/msys64/mingw64/bin/../lib/gcc/avr/8.5.0/device-specs/specs-avr2
COLLECT_GCC=C:\msys64\mingw64\bin\avr-gcc.exe
COLLECT_LTO_WRAPPER=c:/msys64/mingw64/bin/../libexec/gcc/avr/8.5.0/lto-wrapper.exe
Target: avr
Configured with: ../gcc-8.5.0/configure --build=x86_64-w64-mingw32 --prefix=/mingw64 --target=avr --enable-languages=c,c++ --disable-nls --disable-libssp --disable-shared --disable-threads --disable-libgomp --disable-libada --with-dwarf2 --enable-mingw-wildcard
Thread model: single
gcc version 8.5.0 (GCC)

[mcuee]

And I believe WinAVR is no longer supported.

$ make clean
rm -rf *.o *.elf *.lst *.map *.sym *.lss *.eep *.srec *.bin *.hex *.tmp.sh

$ make
avr-gcc (WinAVR 20100110) 4.3.3
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

BAUD RATE CHECK: Desired: 115200, Real: 117647, UBRRL = 16, Difference=2.1%
avr-gcc -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega8 -DF_CPU=16000000L   -DBAUD_RATE=115200 -DLED_START_FLASHES=3               '-DVIRTUAL_BOOT_PARTITION' '-Dsave_vect_num=EE_RDY_vect_num'   -c -o optiboot.o optiboot.c
optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2%
optiboot.c:498: warning: 'leaf' attribute directive ignored
optiboot.c:499: warning: 'leaf' attribute directive ignored
optiboot.c: In function 'main':
optiboot.c:830: error: 'EE_RDY_vect_num' undeclared (first use in this function)
optiboot.c:830: error: (Each undeclared identifier is reported only once
optiboot.c:830: error: for each function it appears in.)
optiboot.c: In function 'read_mem':
optiboot.c:1535: error: 'EE_RDY_vect_num' undeclared (first use in this function)
make: *** [<builtin>: optiboot.o] Error 1

[mcuee]

For macOS, I have encountered issues with Atmel provided toolchain due to macOS security thingy.

I have figured the solution.

• https://github.com/stefanrueger/urboot/issues/8#issuecomment-1327964594
• https://support.blackfire.io/en/articles/3669492-issues-with-macos-catalina

[mcuee]

It would also be good to have some benchmarks about using different toolchains. I've seen some old 2018 threads with some data. It would be good to compare the code size generated using different version of the toolchain, at least the avr-gcc 5.4 based version and 7.3 based version.

Example output for hex files generated by different toolchain using makeall.arduino.sh. We can see that older avr-gcc 5.4.0 based version will usually produce the smallest code size with some exceptions. And avr-gcc 12.1.0 usually produces the biggest code size with a few exceptions.

hexls is a neat utility from urboot project. https://github.com/stefanrueger/urboot/blob/main/src/hexls

1) Using avr-gcc 5.4.0 Atmel toolchain (overall best) 2) Using avr-gcc 7.3, current version of Arduino toolchain or Atmel toolchain 3) MSYS2 mingw64 avr-gcc 8.5.0 (overall second best) 4) avr-gcc 12.1.0. (overall worst)

hex file	Size (5.4.0)	Space (5.4.0)	Size (7.3.0)	Space (7.3.0)	Size (8.5.0)	Space (8.5.0)	Size (12.1.0)	Space (12.1.0)
optiboot_atmega1280.hex	787	1024	789	1024	791	1024	783	1024
optiboot_atmega1284p.hex	784	1024	790	1024	792	1024	788	1024
optiboot_atmega168.hex	474	512	490	512	476	512	494	512
optiboot_atmega168p_16MHz.hex	474	512	490	512	476	512	494	512
optiboot_atmega168p_UART__.hex	474	512	490	512	476	512	494	512
optiboot_atmega32.hex	458	512	466	512	470	512	484	512
optiboot_atmega328.hex	474	512	490	512	476	512	494	512
optiboot_atmega328_pro_8MHz.hex	474	512	490	512	476	512	494	512
optiboot_atmega8.hex	458	512	466	512	470	512	484	512
optiboot_atmega88.hex	474	512	490	512	476	512	494	512
optiboot_bobuino.hex	784	1024	790	1024	792	1024	788	1024
optiboot_diecimila.hex	474	512	490	512	476	512	494	512
optiboot_lilypad.hex	474	512	490	512	476	512	494	512
optiboot_luminet.hex	556	576	564	576	564	576	572	576
optiboot_pro_16MHz.hex	474	512	490	512	476	512	494	512
optiboot_pro_20mhz.hex	474	512	490	512	476	512	494	512
optiboot_pro_8MHz.hex	474	512	490	512	476	512	494	512
optiboot_sanguino.hex	747	1024	749	1024	751	1024	747	1024
optiboot_wildfirev2.hex	785	1024	791	1024	793	1024	789	1024
optiboot_xplained168pb.hex	474	512	490	512	476	512	494	512
optiboot_xplained328p.hex	474	512	490	512	476	512	494	512
optiboot_xplained328pb.hex	474	512	490	512	476	512	494	512

[mcuee]

The older toolchain does not work with the custom linker script, so I checked out one commit earlier than that (https://github.com/Optiboot/optiboot/commit/68b7a55b1e65e57dc822e2d52c31d2da3c467b8f)

This may not be comparable with the above git main results though.

1) Using avr-gcc 4.8.1 from urboot under Linux. 2) Using the 4.9.2 toolchain from urboot under Linux. 3) Using avr-gcc 5.4.0 Atmel toolchain (overall best) 4) avr-gcc 7.3 Current version of Arduino toolchain or Atmel toolchain 5) MSYS2 mingw64 avr-gcc 8.5.0 (overall second best) 6) avr-gcc 12.1.0. (overall worst)

hex file	Size (4.8.1)	Space (4.8.1)	Size (4.9.2)	Space (4.9.2)	Size (5.4.0)	Space (5.4.0)	Size (7.3.0)	Space (7.3.0)	Size (8.5.0)	Space (8.5.0)	Size (12.1.0)	Space (12.1.0)
optiboot_atmega1280.hex	818	1024	823	1024	797	1024	795	1024	798	1024	792	1024
optiboot_atmega1284p.hex	816	1024	820	1024	792	1024	796	1024	798	1024	794	1024
optiboot_atmega168.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega168p_16MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega168p_UART__.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega32.hex	470	512	466	512	464	512	472	512	476	512	490	512
optiboot_atmega328.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega328_pro_8MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_atmega8.hex	472	512	468	512	466	512	472	512	476	512	490	512
optiboot_atmega88.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_bobuino.hex	816	1024	820	1024	792	1024	796	1024	798	1024	794	1024
optiboot_diecimila.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_lilypad.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_luminet.hex	570	640	570	640	564	640	570	640	570	640	578	640
optiboot_pro_16MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_pro_20mhz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_pro_8MHz.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_sanguino.hex	776	1024	775	1024	755	1024	755	1024	758	1024	754	1024
optiboot_wildfirev2.hex	817	1024	821	1024	793	1024	797	1024	800	1024	796	1024
optiboot_xplained168pb.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_xplained328p.hex	490	512	488	512	482	512	496	512	482	512	500	512
optiboot_xplained328pb.hex	490	512	488	512	482	512	496	512	482	512	500	512

7) WinAVR gcc version 4.3.3 (WinAVR 20100110) can only build a few hex files.

$ ./hexls
856 1024 o8.2 -.s-.-r-- optiboot_atmega1280.hex
856 1024 o8.2 -.s-.-r-- optiboot_atmega1284p.hex
494 512 o8.2 -.s-.-r-- optiboot_atmega32.hex
494 512 o8.2 -.s-.-r-- optiboot_atmega8.hex
856 1024 o8.2 -.s-.-r-- optiboot_bobuino.hex
816 1024 o8.2 -.s-.-r-- optiboot_sanguino.hex
857 1024 o8.2 -.s-.-r-- optiboot_wildfirev2.hex

[mcuee]

Nice refernce from here, back in August 2018. https://github.com/Optiboot/optiboot/pull/142#issuecomment-412324592

Atmel avr-gcc 5.4.0 based toolchain was the winner at that time. I think it is still the overall best now.

target \ compiler	1.0.6	1.6.9	1.6.13	1.8.5	microchip avr-gcc 5.4
atmega128	672	654	666	666	642
atmega1280	680	666	680	680	654
atmega1281	684	666	678	678	652
atmega1284	682	662	674	674	646
atmega1284p	682	662	674	674	646
atmega16	474	436	434	434	432
atmega162	478	434	432	432	426
atmega164a	x	456	454	454	448
atmega164p	x	456	454	454	448
atmega168	502	456	454	454	448
atmega168p	502	456	454	454	448
atmega168pb LED=B5	x	456	454	454	448
atmega169	500	454	452	452	446
atmega169p	500	454	452	452	446
atmega2560	684	666	678	678	652
atmega2561	684	666	678	678	652
atmega32	478	438	434	434	432
atmega324a	x	456	454	454	448
atmega324p	x	456	454	454	448
atmega324pa	x	454	452	452	446
atmega324pb	x	x	x	x	x
atmega328	502	456	454	454	448
atmega328 BIGBOOT=1	590	576	578	578	570
atmega328 LED_START_FLASHES=0	456	410	410	410	404
atmega328 LED_START_FLASHES=0 LED_START_ON=1 SUPPORT_EEPROM=1	x	512	x	x	508
atmega328 LED_START_FLASHES=0 SUPPORT_EEPROM=1	x	508	512	512	504
atmega328 SOFT_UART=1	502	458	458	458	452
atmega328_pro8	502	456	454	454	448
atmega328pb LED=B5	x	x	x	x	x
atmega329	498	454	454	454	448
atmega3290	502	456	454	454	448
atmega3290p	502	456	454	454	448
atmega329p	498	456	454	454	448
atmega64	632	610	618	618	598
atmega640	x	662	674	674	648
atmega644p	644	620	628	628	608
atmega644p	644	620	628	628	608
atmega649	638	614	624	624	604
atmega6490	642	618	626	626	606
atmega6490p	x	618	626	626	606
atmega649p	x	618	626	626	606
atmega8	474	436	432	432	430
atmega8515	x	438	434	434	432
atmega8535	x	438	434	434	432
atmega88	500	456	454	454	448
atmega88p LED=B5	500	456	454	454	448
atmega88pb LED=B5	x	456	454	454	448
attiny1634	x	546	546	546	536
attiny828	x	x	x	x	x
attiny84	660	546	546	546	548
attiny841	x	580	578	578	576
bobuino	682	662	674	674	646
diecimila	502	456	454	454	448
lilypad	502	456	454	454	448
lilypad_resonator	502	456	454	454	448
luminet	660	546	546	546	548
mega1280	680	666	680	680	654
mega2560	684	666	678	678	652
mighty1284	682	662	674	674	646
pro16	502	456	454	454	448
pro20	502	456	454	454	448
pro8	502	456	454	454	448
sanguino	644	620	628	628	608
virboot328	x	570	570	570	560
virboot8	x	572	568	568	570
wildfire	682	662	674	674	646
wildfirev2	682	662	674	674	646
wildfirev3	682	662	674	674	646
xplained168pb	502	456	454	454	448
xplained328p	502	456	454	454	448
xplained328pb	502	456	454	454	448

[mcuee]

@WestfW @MCUdude and @stefanrueger

Maybe you will be interested in the results above.

Interestingly the conclusion is a bit different from urboot test results below. For urboot, 4.8.1 performs better than 5.4.0.

• https://github.com/stefanrueger/urboot/issues/8#issuecomment-1327970048

[hecko]

Im happy to help here in some capacity, however It seems that at the moment there is way too many combinations of compilers , compiler versions, environments and target platforms to make some baseline for CI/CD. Im kind of inclined to suggest having one or two environments fully officially supported and clean-up the whole repo, Makefiles etc.

The Microchip version of gcc toolchain for Linux seems like one of those envs that make sense to be officially supported by Optiboot. Im not sure about the other ones as it seems they lack some uC support that Microchip version has (like atmega328pb).

I have made a reference Dockerfile for compiling against Microchip toolchain in https://github.com/hecko/optiboot_docker_build - this can be easily ran using GitHub Actions and hex files can be automagically published / analyzed somewhere.

Someone authoritative should really make a decision here. Especially now that HEAD version of Optiboot has different compilation options to the 8.0 version that will definitely confuse many people / compilation systems once 9.0 or whatever is released.

[mcuee]

Someone authoritative should really make a decision here. Especially now that HEAD version of Optiboot has different compilation options to the 8.0 version that will definitely confuse many people / compilation systems once 9.0 or whatever is released.

@WestfW It seems that the current git head is of version 8.3, just wondering if you want release it as Optiboot 8.3. That may help to reduce the confusions.

[mcuee]

@hecko

There is actually Travis-CI in the repo but it seems that it is not utlized. Maybe you can model that for your Docker Container or even better create a PR for github action. https://github.com/Optiboot/optiboot/blob/master/.travis.yml

Your Dockerfile works great -- just tested it under Ubuntu 20.04 on my mini PC.

mcuee@HeroBox:~/build/optiboot_docker_build$ BUILDKIT_PROGRESS=plain DOCKER_BUILDKIT=1 docker build --no-cache --output out .
...
....
#14 [build_stage 11/20] WORKDIR /opt
#14 sha256:b0589afa15d5711690db901b41924603ce146870dff7b51b80900c88229788fd
#14 DONE 0.0s

#15 [build_stage 12/20] RUN git clone https://github.com/Optiboot/optiboot.git
#15 sha256:7969cd2e2d0820e39aab8be73ac71f04d2612f347d73c74b31506be8ccfae058
#15 0.588 Cloning into 'optiboot'...
#15 DONE 2.0s

#16 [build_stage 13/20] WORKDIR /opt/optiboot/optiboot/bootloaders/optiboot
#16 sha256:ab81a4c854c83acfe709c0a22e7396ea73141e5c0dcfc7d40c83c1d3899789f7
#16 DONE 0.1s

#17 [build_stage 14/20] RUN git reset --hard 55d1e6b36922e4b8e3a32e6cea8ec03127ed65bf
#17 sha256:3b7667d40bebc010a7575e1e745f379b4ae7f3412d99a51752cdb3abd6a554d2
#17 0.576 HEAD is now at 55d1e6b Prevent use of "PINx |= bit" when toggling an LED on a not-IOSPACE port.
#17 DONE 0.6s

#18 [build_stage 15/20] RUN ../../../../avr8-gnu-toolchain-linux_x86_64/bin/avr-gcc -B ../../../../atpack/gcc/dev/atmega328p/ -I ../../../../atpack/include/ -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega328p -DF_CPU=8000000L -DBAUD_RATE=57600 -DBIGBOOT=1 -DSUPPORT_EEPROM=1 -DLED=D5 -DLED_START_FLASHES=3 -DRS485=C3 -DWDTTIME=8 -DNO_START_APP_ON_POR=1 -c -o optiboot.o optiboot.c
#18 sha256:fddafafc5180a4b49a70345b413fef636a4ce8e329dd654c2c445ee5df4fd11a
#18 0.623 optiboot.c:429:2: warning: #warning BAUD_RATE off by greater than 2% [-Wcpp]
#18 0.623  #warning BAUD_RATE off by greater than 2%
#18 0.623   ^~~~~~~
#18 DONE 0.7s

#19 [build_stage 16/20] RUN ../../../../avr8-gnu-toolchain-linux_x86_64/bin/avr-gcc -B ../../../../atpack/gcc/dev/atmega328p/ -I ../../../../atpack/include/ -g -Wall -Os -fno-split-wide-types -mrelax -mmcu=atmega328p -DF_CPU=8000000L -DBAUD_RATE=57600 -DBIGBOOT=1 -DSUPPORT_EEPROM=1 -DLED=D5 -DLED_START_FLASHES=3 -DRS485=C3 -DWDTTIME=8 -DNO_START_APP_ON_POR=1 -Wl,--section-start=.text=0x7c00 -Wl,--section-start=.version=0x7ffe -Wl,--relax -nostartfiles -o optiboot.elf optiboot.o
#19 sha256:6b19fd20bcdd529fce43a015f82ec9427dc4f3117d1183e446c250341d61b91a
#19 DONE 0.7s

#20 [build_stage 17/20] RUN ../../../../avr8-gnu-toolchain-linux_x86_64/bin/avr-objcopy -j .text -j .data -j .version --set-section-flags .version=alloc,load -O ihex optiboot.elf optiboot.hex
#20 sha256:4918d30a1d5dff4efbdae20560580f5b9563ca4c9d4d11c3a2c90f2c4edae9d4
#20 DONE 0.6s

#21 [build_stage 18/20] RUN ../../../../avr8-gnu-toolchain-linux_x86_64/bin/avr-objdump -h -S optiboot.elf > optiboot.lst
#21 sha256:f5efe296c1b25c173fc6d533ee281a5bbd77dfcd4d58395291607313c9fe6193
#21 DONE 0.6s

#22 [build_stage 19/20] RUN ../../../../avr8-gnu-toolchain-linux_x86_64/bin/avr-objdump -d optiboot.elf > optiboot.asm
#22 sha256:fce7213b6f3612a14fc02d1b5471b8b5fb705a303349fa1300419d19b28828d4
#22 DONE 0.6s

#23 [build_stage 20/20] RUN ../../../../avr8-gnu-toolchain-linux_x86_64/bin/avr-size -C optiboot.elf
#23 sha256:334e6d7d0a9efe9e16781b1efb1069f2fd36c4ceaf0716655ef641642ffcf45a
#23 0.476 AVR Memory Usage
#23 0.476 ----------------
#23 0.476 Device: Unknown
#23 0.476
#23 0.476 Program:     769 bytes
#23 0.476 (.text + .data + .bootloader)
#23 0.476
#23 0.476 Data:          0 bytes
#23 0.476 (.data + .bss + .noinit)
#23 0.476
#23 0.476
#23 DONE 0.5s

#24 [export_stage 1/3] COPY --from=build_stage /opt/optiboot/optiboot/bootloaders/optiboot/optiboot.hex .
#24 sha256:1e0f2446c40d54c27c1e7d948f5e6d5d53d3ce1fd5a1e0ceea449ceffe753224
#24 DONE 0.2s

#25 [export_stage 2/3] COPY --from=build_stage /opt/optiboot/optiboot/bootloaders/optiboot/optiboot.lst .
#25 sha256:744307a6d9496686908ab3b1ba3a3b3fd3fc47560cd7746378ea63c46bb093e1
#25 DONE 0.1s

#26 [export_stage 3/3] COPY --from=build_stage /opt/optiboot/optiboot/bootloaders/optiboot/optiboot.asm .
#26 sha256:42314c00a0c7acfdc0a7adbbd10a6108681461c487c129a5e522e3d1cfe4d15d
#26 DONE 0.1s

#27 exporting to client
#27 sha256:b60a1292d407630dbb741f28ab6ea4ce3cca872ac28eeee56f4e66a182eca4bc
#27 copying files
#27 copying files 46.73kB 0.0s done
#27 DONE 0.0s
mcuee@HeroBox:~/build/optiboot_docker_build$ cat out/optiboot.hex
:107C000001C009C1112484B7882359F083FF09C03A
:107C1000982F937011F097EF94BF282E80E0E6D054
:107C2000EFC185E08093810082E08093C00088E10D
:107C30008093C10086E08093C20080E18093C400FD
:107C40003B9A439889E2D2D0559A86E028E13EEFEC
:107C500091E0309385002093840096BBB09BFECFCB
:107C60004D9AA8954091C00047FD02C0815089F708
:107C700023E0D22ECC24C39435E0B32E41E1A42ED0
:107C8000A9D0813459F4A6D0182FB6D083E011388A
:107C900009F488E08FD080E18DD0F2CF823419F4DE
:107CA00084E1B2D0F8CF853411F485E0FACF853580
:107CB00041F490D0C82F8ED0D82FCC0FDD1F9CD090
:107CC000EACF863521F484E09FD080E0E3CF84368C
:107CD00009F045C07FD07ED0182F7CD0082F812C92
:107CE0009924939474018FEFE81AF80A73D0F40181
:107CF000808347011E11F6CF7FD0053491F4E12E29
:107D0000F12C10E000E0F801F3950E151F0509F4C1
:107D1000C2CF6081C8018C0F9D1F95D00F5F1F4F90
:107D2000F2CFFE01D7BEE89507B600FCFDCFFE01FD
:107D3000A0E0B1E0CD0102962D913C910901C7BEB2
:107D4000E89511243296DC011813F4CFFE01B7BE7A
:107D5000E89507B600FCFDCFA7BEE8959CCF843719
:107D6000D1F438D037D0182F35D0082F45D07E0128
:107D7000053451F4C7015FD01DD01150FFEFEF1A49
:107D8000FF0A1111F7CF87CFF70185917F0112D03C
:107D90001150D1F780CF853739F42ED08EE10AD03B
:107DA00085E908D08FE076CF813509F088CF88E06B
:107DB0001DD085CF9091C00095FFFCCF906490932B
:107DC000C000439A8093C6008091C00086FFFCCF1C
:107DD000439808958091C00087FFFCCF8091C00038
:107DE00084FD01C0A8958091C6000895E0E6F0E00A
:107DF00098E1908380830895EDDF803219F088E068
:107E0000F5DFFFCF84E1D6CFCF93C82FE3DFC1509A
:107E1000E9F7CF91F1CFFC010A0167BFE895112482
:107E200007B600FCFDCF667029F0452B19F481E1FF
:107E300087BFE8950895F999FECF92BD81BDF89A64
:107E4000992780B50895262FF999FECF1FBA92BDC4
:107E500081BD20BD0FB6F894FA9AF99A0FBE01962B
:107E60000895FF56657273696F6E3D382E33004476
:107E700065766963653D61746D656761333238703D
:107E800000465F4350553D383030303030304C0084
:107E9000424947424F4F543D31004275696C743A34
:107EA00044656320323220323032323A30303A3454
:107EB000353A313400554152543D30004241554429
:107EC0005F524154453D35373630300052533438D7
:107ED000353D433300535550504F52545F45455044
:107EE000524F4D3D31004C45443D4435004C4544D6
:107EF0005F53544152545F464C41534845533D33C0
:017F00000080
:027FFE00030876
:0400000300007C007D
:00000001FF

[mcuee]

@WestfW

If you look at @hecko's Dockerfile, it actually uses Microchip/Atmel DFPs. This should sort out your comments here.

• https://github.com/Optiboot/optiboot/pull/257#issuecomment-428436417

On the other hand, I htink the DFPs are meant to be used with Atmel toolchain based on avr-gcc 7.3.0, which does produce binaries with bigger code size than the older toolchain based on avr-gcc 5.4.0. But at least we can use the older 5.4.0 for most of the build and only use 7.3.0 for the chips 5.4.0 does not support.

urboot uses this strategy -- it uses 4.8.1 or 5.4.0 based on certain conditions. https://github.com/stefanrueger/urboot/blob/main/src/Makefile

[hecko]

@mcuee @WestfW As suggested I have now created a simple GitHub action to have a frame of reference whether the builds for Arduinos pass the make against stock Debian 11 avr gcc and avr clibs. Please see https://github.com/Optiboot/optiboot/pull/362 and comment if possible.

[mcuee]

362 is pretty good, the only thing is that I will prefer to use Microchip toolchain based on avr-gcc 5.4.0 based on the code size tests mentioned earlier in this issue.

[mcuee]

@hecko

If #362 got positive review from @WestfW, it will be good to futher the work to the build of optiboot_x which will require the use the Microchip DFPs if we use Microchip toolchain based on avr-gcc 7.3.0.

[hecko]

I have created another workflow for the Microchip toolchain building all targets by running makeall.everycpu.sh, but Im unhappy about the Makefiles overdrive directive for CFLAGS as the environmental variables are not accepted and I havent found a quick way to integrrate the -I and -B flags into the process quickly

Please have a look at this change what you think @mcuee https://github.com/hecko/optiboot/commit/61897771a81fb5279c2a6d2a0570cc37da2358c9

[mcuee]

@hecko

Pretty impressive fast move. Thanks.

The main missing thing is the archive of the resultant files now.

BTW, makeall.everycpu.sh does not include newer UPDI AVR chips which is included in makeall.mega0.sh.

[hecko]

Thats the thing @mcuee - are you sure you actually want to be releasing HEX files from this repository or simply run those workflows to make sure the build process still works correctly with new releases of the source codes?

I think having a separate repository (something like optiboot-releases) would make more sense - we can then build the HEX files in this optiboot repo, but push them into a different repo for each official release. Its going to be lots and lots of files and I think having a separate entity for this is better way to go than storing the files in the main optiboot devel repo.

[mcuee]

Thats the thing @mcuee - are you sure you actually want to be releasing HEX files from this repository or simply run those workflows to make sure the build process still works correctly with new releases of the source codes?

Just archive the build artifacts so that the users can try the snapshot if they want, not really meant for release.

I think having a separate repository (something like optiboot-releases) would make more sense - we can then build the HEX files in this optiboot repo, but push them into a different repo for each official release. Its going to be lots and lots of files and I think having a separate entity for this is better way to go than storing the files in the main optiboot devel repo.

I do not think a seperate repo is necessary. When it comes to release, you can always release the hex files as you can see from the past optiboot release. https://github.com/Optiboot/optiboot/releases

[hecko]

so, this is the example of automatically generating hex files from a GitHub action: https://github.com/hecko/optiboot/commit/fdf98b1e826dfa9e287d48bbe8c840899119262e

There needs to be a little cleanup as there are some non-hex, lst and elf files included, but this demonstrates that this can easily be done. Now the question is - if there are more environments in which files are be compiled - like the mentioned two different versions of Microchip toolkit - what do you aim to do about that? Do you want to create two dirs for the generated files depending on the toolchain being used?

[mcuee]

so, this is the example of automatically generating hex files from a GitHub action: hecko@fdf98b1

Nice. It would be good to zip the generated hex/elf/lst files and give the user the option to download the zip file.

Reference from avrdude github action: you can see the last step is to upload the archive. https://github.com/avrdudes/avrdude/blob/main/.github/workflows/build.yml

There needs to be a little cleanup as there are some non-hex, lst and elf files included, but this demonstrates that this can easily be done. Now the question is - if there are more environments in which files are be compiled - like the mentioned two different versions of Microchip toolkit - what do you aim to do about that? Do you want to create two dirs for the generated files depending on the toolchain being used?

Yes that will be nice -- two seperate dirs for two toolchains.

But let's hear from @WestfW as well.

[hecko]

The output for both toolchains as discussed:

https://github.com/hecko/optiboot/commit/8ed018f569388c17e507d5295f27c8d7b366cc69

[WestfW]

But let's hear from @WestfW as well.

I would like the optiboot source repository to remain "uncluttered" by hex and lst files. That's not the same as not checking for correct compilation, though. I am not up-to-date on things like CI and containers and such; they all showed up after my retirement from industry.

I think having a separate repository (something like optiboot-releases) would make more sense

That would be OK. I'm also happy to have the "boards packages" contain the .hex files. But the number of possible combinations that "might be useful" (even for testing) grows very large very quickly (The @MCUDude collection of optiboot hex files is over 10000 individual files!), and I think that's ridiculously bloaty to add to a source repsository (not to mention, as soon as it looks like you're providing .hex files, SOMEONE will complain that the particular one that they want isn't there. ("how about a 328pb .hex, pretending to be a 328p, but one the 2nd UART?")

[mcuee]

I think it is okay to skip the hex file other than the few most common ones in optiboot, then create another repo for the large collection if that is desired to have (personally I do not think it is necessary).

What I was trying to say is to create github action which can generate the big collection of hex files (either automatically after each commit, or only on-demand). Then let the users download the artificat from github action which contains the hex files. But then the hex files are not part of the repo, just provided for convenience sake and not officially supported.

Example from @stefanrueger's urboot project. https://github.com/stefanrueger/urboot

To me it is good enough to have the representative hex files like here. https://github.com/stefanrueger/urboot/tree/main/src/all

But I agree it is too much to have the following big collection of hex files in the source repo. https://github.com/stefanrueger/urboot/tree/main/bootloaders

As for @MCUdude's optiboot_flash, I think the situation is different, he has a few Arduino Cores to support and the hex files are indeed necessary. Of course, it could be possible to split the hex file collections into a seperate repo as well. https://github.com/MCUdude/optiboot_flash

Maybe @stefanrueger has that in mind as well.

@WestfW I am not so sure if anyone is still using this repo with Arduino IDE. I think it is even better to remove the legacy stuff like boards.txt and package.json template.