west signing seems to be broken on windows

[thedjnK]

Describe the bug Trying to build a signed image, after running ninja it fails at the signing step with an error that %1 is not a valid win32 application and no signed image is created

To Reproduce Steps to reproduce the behavior:

1. Build project with signing enabled and setup (I used philospher example, I used ninja menuconfig to enable mcuboot support, add the sample key file and enable outputting a confirmed image)
2. Build with ninja
3. Error shown is:

    [1/1] Linking C executable zephyr\zephyr.elf
    FAILED: zephyr/zephyr.elf zephyr/zephyr.hex zephyr/zephyr.bin zephyr/zephyr.lst zephyr/zephyr.stat zephyr/zephyr.signed.bin zephyr/zephyr.signed.confirmed.bin zephyr/zephyr.signed.hex zephyr/zephyr.signed.confirmed.hex 
    cmd.exe /C "cd . && C:\gcc-arm\bin\arm-none-eabi-gcc.exe   zephyr/CMakeFiles/zephyr_final.dir/misc/empty_file.c.obj zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj -o zephyr\zephyr.elf  -Wl,-T  zephyr/linker_pass_final.cmd  -Wl,-Map=C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr_final.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/mpu/libarch__arm__core__aarch32__cortex_m__mpu.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/cmse/libarch__arm__core__aarch32__cortex_m__cmse.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/serial/libdrivers__serial.a  modules/hal_nordic/libmodules__hal_nordic.a  modules/mcuboot/lib..__bootloader__mcuboot__boot__bootutil__zephyr.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -L"c:/gcc-arm/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v8-m.main/nofp"  -LC:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn && cmd.exe /C "cd /D C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu\zephyr && "C:\Program Files\CMake\bin\cmake.exe" -E rename zephyr_final.map zephyr.map && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=ihex --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.hex && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=binary --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.bin && C:\gcc-arm\bin\arm-none-eabi-objdump.exe -d -S zephyr.elf > zephyr.lst && C:\gcc-arm\bin\arm-none-eabi-readelf.exe -e zephyr.elf > zephyr.stat && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem --pad --confirm""
    Traceback (most recent call last):
      File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "C:\Python37\lib\runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "C:\Python37\lib\site-packages\west\app\main.py", line 806, in <module>
        main()
      File "C:\Python37\lib\site-packages\west\app\main.py", line 779, in main
        app.run(argv or sys.argv[1:])
      File "C:\Python37\lib\site-packages\west\app\main.py", line 106, in run
        self.run_command(argv)
      File "C:\Python37\lib\site-packages\west\app\main.py", line 338, in run_command
        self.run_extension(args.command, argv)
      File "C:\Python37\lib\site-packages\west\app\main.py", line 408, in run_extension
        command.run(args, unknown, self.topdir, manifest=self.manifest)
      File "C:\Python37\lib\site-packages\west\commands.py", line 129, in run
        self.do_run(args, unknown)
      File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 187, in do_run
        signer.sign(self, build_dir, bcfg, formats)
      File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 278, in sign
        subprocess.check_call(sign_bin)
      File "C:\Python37\lib\subprocess.py", line 336, in check_call
        retcode = call(*popenargs, **kwargs)
      File "C:\Python37\lib\subprocess.py", line 317, in call
        with Popen(*popenargs, **kwargs) as p:
      File "C:\Python37\lib\subprocess.py", line 769, in __init__
        restore_signals, start_new_session)
      File "C:\Python37\lib\subprocess.py", line 1172, in _execute_child
        startupinfo)
    OSError: [WinError 193] %1 is not a valid Win32 application
    ninja: build stopped: subcommand failed.

Expected behavior The application to be signed

Impact Showstopper

Environment (please complete the following information):

• OS: Windows 7 64-bit
• Toolchain: GCC ARM 9
• Commit: 8925af94f292302b39ec749eb98ef3bee5c7389f
• West: 0.9.0
• Python: 3.7

[carlescufi]

@thedjnK what exact commands did you run? Could you please document those here so that we can try to reproduce?

[mbolivar-nordic]

@thedjnK I think what's going on here is that you neither have imgtool installed via pip3 nor have the registry setup to run .py files as executables.

Without imgtool installed from pip3, west sign will run the imgtool.py script in the mcuboot repository. If Windows is not configured to run the correct interpreter for a .py file, you'll get this error.

python.org says that the default installer should make the right file associations: https://docs.python.org/3.9/using/windows.html#from-file-associations

However, that's not working on my Windows setup, either.

I'll think about how to fix this, but in the meantime, you have these workarounds:

1. pip3 install imgtool
2. provide --tool-path and point it at a .bat script that runs imgtool.py in your mcuboot repo
3. put an imgtool script in your PATH that runs imgtool.py in your mcuboot repo

[mbolivar-nordic]

@nashif consider changing to priority low based on the above diagnosis

[carlescufi]

@nashif consider changing to priority low based on the above diagnosis

Done now.

[thedjnK]

@mbolivar-nordic I've checked and python files are associated with python 3.7, if I create an empty .py file and run it, I get the python console open and close very quickly. I didn't have imgtool installed via pip as I assumed it would use the bundled version with zephyr as it used to do in zephyr 1.x, so I've installed imgtool via pip and I'm still getting the same result:

C:\pure_zephyr\zephyr>pip install imgtool
Collecting imgtool
  Downloading https://files.pythonhosted.org/packages/6f/bb/712f20549ab5a82571009eb4fef9b6f0b71a4685fa28bfaa9c1cbabd6d17/imgtool-1.7.1-py3-none-any.whl
Requirement already satisfied: cbor>=1.0.0 in c:\python37\lib\site-packages (from imgtool) (1.0.0)
Requirement already satisfied: click in c:\python37\lib\site-packages (from imgtool) (6.6)
Requirement already satisfied: intelhex>=2.2.1 in c:\python37\lib\site-packages (from imgtool) (2.2.1)
Requirement already satisfied: cryptography>=2.4.2 in c:\python37\lib\site-packages (from imgtool) (2.9.2)
Requirement already satisfied: six>=1.4.1 in c:\python37\lib\site-packages (from cryptography>=2.4.2->imgtool) (1.12.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in c:\python37\lib\site-packages (from cryptography>=2.4.2->imgtool) (1.14.0)
Requirement already satisfied: pycparser in c:\python37\lib\site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.4.2->imgtool) (2.20)
Installing collected packages: imgtool
Successfully installed imgtool-1.7.1
WARNING: You are using pip version 19.2.2, however version 21.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

C:\pure_zephyr\zephyr>cd C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu

C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu>ninja
[1/1] Linking C executable zephyr\zephyr.elf
FAILED: zephyr/zephyr.elf zephyr/zephyr.hex zephyr/zephyr.bin zephyr/zephyr.lst
zephyr/zephyr.stat zephyr/zephyr.signed.bin zephyr/zephyr.signed.confirmed.bin zephyr/zephyr.signed.hex zephyr/zephyr.signed.confirmed.hex
cmd.exe /C "cd . && C:\gcc-arm\bin\arm-none-eabi-gcc.exe   zephyr/CMakeFiles/zephyr_final.dir/misc/empty_file.c.obj zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj -o zephyr\zephyr.elf  -Wl,-T  zephyr/linker_pass_final.cmd  -Wl,-Map=C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr_final.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/mpu/libarch__arm__core__aarch32__cortex_m__mpu.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/cmse/libarch__arm__core__aarch32__cortex_m__cmse.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/serial/libdrivers__serial.a  modules/hal_nordic/libmodules__hal_nordic.a  modules/mcuboot/lib..__bootloader__mcuboot__boot__bootutil__zephyr.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -L"c:/gcc-arm/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v8-m.main/nofp"  -LC:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m33  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn && cmd.exe /C "cd /D C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu\zephyr && "C:\Program Files\CMake\bin\cmake.exe" -E rename zephyr_final.map zephyr.map && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=ihex --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.hex && C:\gcc-arm\bin\arm-none-eabi-objcopy.exe --gap-fill 0xff --output-target=binary --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.bin && C:\gcc-arm\bin\arm-none-eabi-objdump.exe -d -S zephyr.elf > zephyr.lst && C:\gcc-arm\bin\arm-none-eabi-readelf.exe -e zephyr.elf > zephyr.stat && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem && C:\Python37\python.exe -m west.app.main sign --quiet --tool imgtool --tool-path C:/pure_zephyr/bootloader/mcuboot/scripts/imgtool.py --build-dir C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu --bin --sbin C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.bin --hex --shex C:/pure_zephyr/zephyr/samples/philosophers/build_5340_cpu/zephyr/zephyr.signed.confirmed.hex -- --key C:/pure_zephyr/bootloader/mcuboot/root-rsa-2048.pem --pad --confirm""
Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37\lib\site-packages\west\app\main.py", line 806, in <module>
    main()
  File "C:\Python37\lib\site-packages\west\app\main.py", line 779, in main
    app.run(argv or sys.argv[1:])
  File "C:\Python37\lib\site-packages\west\app\main.py", line 106, in run
    self.run_command(argv)
  File "C:\Python37\lib\site-packages\west\app\main.py", line 338, in run_command
    self.run_extension(args.command, argv)
  File "C:\Python37\lib\site-packages\west\app\main.py", line 408, in run_extension
    command.run(args, unknown, self.topdir, manifest=self.manifest)
  File "C:\Python37\lib\site-packages\west\commands.py", line 129, in run
    self.do_run(args, unknown)
  File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 187, in do_run
    signer.sign(self, build_dir, bcfg, formats)
  File "C:\pure_zephyr\zephyr\scripts\west_commands\sign.py", line 278, in sign
    subprocess.check_call(sign_bin)
  File "C:\Python37\lib\subprocess.py", line 336, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Python37\lib\subprocess.py", line 317, in call
    with Popen(*popenargs, **kwargs) as p:
  File "C:\Python37\lib\subprocess.py", line 769, in __init__
    restore_signals, start_new_session)
  File "C:\Python37\lib\subprocess.py", line 1172, in _execute_child
    startupinfo)
OSError: [WinError 193] %1 is not a valid Win32 application
ninja: build stopped: subcommand failed.

C:\pure_zephyr\zephyr\samples\philosophers\build_5340_cpu>imgtool version
1.7.1

[mbolivar-nordic]

Did you try with a pristine build? I can no longer reproduce the error on my windows machine after installing imgtool.

[thedjnK]

Exact commands were:

1. mkdir build
2. cd build
3. cmake -GNinja -DBOARD= ..
4. ninja menuconfig
5. mcuboot support was selected and confirmed image was selected, the mcuboot key file was also added and configuration was saved
6. ninja
7. build failed with above error

[mbolivar-nordic]

I'm building hello_world successfully with this command after running pip3 install imgtool:

west build -b nrf52dk_nrf52832 -s zephyr/samples/hello_world -d build-hello-signed -- -DCONFIG_BOOTLOADER_MCUBOOT=y -DCONFIG_MCUBOOT_SIGNATURE_KEY_FILE=\"bootloader/mcuboot/root-rsa-2048.pem\"

The above comes from the west sign docs here:

https://docs.zephyrproject.org/latest/guides/west/sign.html

Can you try that, to see if perhaps it's a Kconfig issue?

[thedjnK]

@mbolivar-nordic Ah yes, after clearing the build directory and re-generating it, it's now building and signing, thank you

[mbolivar-nordic]

@thedjnK could you please test that #31890 fixes your issue even with imgtool uninstalled (pip3 uninstall imgtool) with a pristine build directory?

[thedjnK]

@mbolivar-nordic tested and that patch resolves the issue, thank you

[mbolivar-nordic]

Thanks for testing!