search

zmkfirmware / zmk

ZMK Firmware Repository
https://zmk.dev/
MIT License
2.83k stars 2.86k forks source link

GitHub Actions build process does not respect local config overlay and uses ZMK global overlay #1382

Open perrwa opened 2 years ago

perrwa commented 2 years ago

I specified a zmk-config/config/reviung41.overlay to use with Cyril's low-profile Reviung41. After building and loading firmware, the keyboard behaves like the original Reviung41 without any of the new overlay changes. My right half top row (B trace) keys do not register and my bottom row behaves like the right half top row should.

My build log: https://github.com/perrwa/zmk-config/runs/7284698921?check_suite_focus=true#step:9:27

Cyril's build log: https://github.com/cyril279/zmk-config-rev41_1350/runs/7237518279?check_suite_focus=true#step:9:27

Process to reproduce:

Hypothesis: During build, the overlay file from local config is used after the global file is applied. I would expect any local config files to supersede and take precedence.

caksoylar commented 2 years ago

To add another data point, this seems to make it impossible (or very hard?) to modify the overlay of only one half of a split keyboard. Say you want to override a GPIO pin for only one half of a cradio shield: You would add the file config/cradio_left.overlay which would override of the kscan node using &kscan0 { input-gpios = <...>; };. However if you do that now, the kscan0 node won't be found because it's not defined yet. (For non-splits you can put the override in the keymap, but it applies to both halves for a split keyboard, as far as I know.)

The order of the overlays picked up in the build are the following:

> west build -b nice_nano_v2 -d build/cradio_left -- -DSHIELD=cradio_left -DZMK_CONFIG=/home/user/zmk-config/config
-- west build: generating a build system
Including boilerplate (Zephyr base (cached)): /home/user/zmk/zephyr/cmake/app/boilerplate.cmake
-- Application: /home/user/zmk/app
-- Adding ZMK config directory as board root: /home/user/zmk-config/config
-- Adding /home/user/zmk/app/boards/shields/cradio
CMake Warning at cmake/ZephyrBuildConfig.cmake:102 (message):
  Didn't find cradio_left
Call Stack (most recent call first):
  /home/user/zmk/zephyr/cmake/app/boilerplate.cmake:113 (find_package)
  /home/user/zmk/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:24 (include)
  /home/user/zmk/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:40 (include_boilerplate)
  CMakeLists.txt:15 (find_package)

-- ZMK Config directory: /home/user/zmk-config/config
-- ZMK Config devicetree overlay: /home/user/zmk-config/config/cradio_left.overlay
-- Using keymap file: /home/user/zmk/app/boards/shields/cradio/cradio.keymap
-- Zephyr version: 3.0.0 (/home/user/zmk/zephyr)
-- Found west (found suitable version "0.13.1", minimum required is "0.7.1")
-- Board: nice_nano_v2, Shield(s): cradio_left
-- Cache files will be written to: /home/user/.cache/zephyr
-- ZEPHYR_TOOLCHAIN_VARIANT not set, trying to locate Zephyr SDK
-- Found host-tools: zephyr 0.13.2 (/home/user/.local/zephyr-sdk-0.13.2)
-- Found dtc: /home/user/.local/zephyr-sdk-0.13.2/sysroots/x86_64-pokysdk-linux/usr/bin/dtc (found suitable version "1.6.0", minimum required is "1.4.6")
-- Found toolchain: zephyr 0.13.2 (/home/user/.local/zephyr-sdk-0.13.2)
-- Found BOARD.dts: /home/user/zmk/app/boards/arm/nice_nano/nice_nano_v2.dts
-- Found devicetree overlay: /home/user/zmk-config/config/cradio_left.overlay
-- Found devicetree overlay: /home/user/zmk/app/boards/shields/cradio/cradio_left.overlay
-- Found devicetree overlay: /home/user/zmk/app/boards/shields/cradio/cradio.keymap
devicetree error: /home/user/zmk-config/config/cradio_left.overlay:1 (column 1): parse error: undefined node label 'kscan0'

The issue I see is that -- Found devicetree overlay: /home/user/zmk-config/config/cradio_left.overlay happens before -- Found devicetree overlay: /home/user/zmk/app/boards/shields/cradio/cradio_left.overlay.

lesshonor commented 2 years ago

As I understand it, user .conf files are also intended to take precedence over the default board/shield .conf files? If said understanding is correct, this issue impacts user .conf files as well as user .overlay files. It is a problem for local builds as well as GH actions.

e.g. this nice!nano MurphPad build:

grep -E 'ZMK_RGB|WS2812' ~/osrc/zmk-src/murphpad-zmk-config/config/murphpad.conf

CONFIG_ZMK_RGB_UNDERGLOW=n
CONFIG_WS2812_STRIP=n

west build -p -b nice_nano -- -DSHIELD=murphpad -DZMK_CONFIG=~/osrc/zmk-src/murphpad-zmk-config/config/

-- west build: making build dir /home/less/osrc/zmk-src/zmk/app/build pristine
-- west build: generating a build system
Including boilerplate (Zephyr base): /home/less/osrc/zmk-src/zmk/zephyr/cmake/app/boilerplate.cmake
-- Application: /home/less/osrc/zmk-src/zmk/app
-- Adding /home/less/osrc/zmk-src/zmk/app/boards/shields/murphpad
-- ZMK Config directory: /home/less/osrc/zmk-src/murphpad-zmk-config/config/
-- ZMK Config Kconfig: /home/less/osrc/zmk-src/murphpad-zmk-config/config//murphpad.conf
-- Using keymap file: /home/less/osrc/zmk-src/murphpad-zmk-config/config//murphpad.keymap
-- Using keymap file: /home/less/osrc/zmk-src/murphpad-zmk-config/config//murphpad.keymap
-- Zephyr version: 3.0.0 (/home/less/osrc/zmk-src/zmk/zephyr)
-- Found Python3: /usr/bin/python3.10 (found suitable exact version "3.10.5") found components: Interpreter 
-- Found west (found suitable version "0.13.1", minimum required is "0.7.1")
-- Board: nice_nano, Shield(s): murphpad
-- Cache files will be written to: /home/less/.cache/zephyr
-- ZEPHYR_TOOLCHAIN_VARIANT not set, trying to locate Zephyr SDK
-- Found host-tools: zephyr 0.13.2 (/home/less/.local/zephyr-sdk-0.13.2)
-- Found dtc: /home/less/.local/zephyr-sdk-0.13.2/sysroots/x86_64-pokysdk-linux/usr/bin/dtc (found suitable version "1.6.0", minimum required is "1.4.6")
-- Found toolchain: zephyr 0.13.2 (/home/less/.local/zephyr-sdk-0.13.2)
-- Found BOARD.dts: /home/less/osrc/zmk-src/zmk/app/boards/arm/nice_nano/nice_nano.dts
-- Found devicetree overlay: /home/less/osrc/zmk-src/zmk/app/boards/shields/murphpad/murphpad.overlay
-- Found devicetree overlay: /home/less/osrc/zmk-src/zmk/app/boards/shields/murphpad/boards/nice_nano.overlay
-- Found devicetree overlay: /home/less/osrc/zmk-src/murphpad-zmk-config/config/murphpad.keymap
-- Generated zephyr.dts: /home/less/osrc/zmk-src/zmk/app/build/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: /home/less/osrc/zmk-src/zmk/app/build/zephyr/include/generated/devicetree_unfixed.h
-- Generated device_extern.h: /home/less/osrc/zmk-src/zmk/app/build/zephyr/include/generated/device_extern.h
-- Including generated dts.cmake file: /home/less/osrc/zmk-src/zmk/app/build/zephyr/dts.cmake
Parsing /home/less/osrc/zmk-src/zmk/app/Kconfig
Loaded configuration '/home/less/osrc/zmk-src/zmk/app/boards/arm/nice_nano/nice_nano_defconfig'
Merged configuration '/home/less/osrc/zmk-src/zmk/app/prj.conf'
Merged configuration '/home/less/osrc/zmk-src/murphpad-zmk-config/config//murphpad.conf'
Merged configuration '/home/less/osrc/zmk-src/zmk/app/boards/shields/murphpad/murphpad.conf'
Merged configuration '/home/less/osrc/zmk-src/zmk/app/boards/shields/murphpad/boards/nice_nano.conf'
Configuration saved to '/home/less/osrc/zmk-src/zmk/app/build/zephyr/.config'
Kconfig header saved to '/home/less/osrc/zmk-src/zmk/app/build/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/less/.local/zephyr-sdk-0.13.2/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Configuring done

Yet running grep -E 'ZMK_RGB|WS2812' build/zephyr/.config post-build...

CONFIG_ZMK_RGB_UNDERGLOW=y
CONFIG_ZMK_RGB_UNDERGLOW_EXT_POWER=y
CONFIG_ZMK_RGB_UNDERGLOW_BRT_MIN=0
CONFIG_ZMK_RGB_UNDERGLOW_BRT_MAX=100
CONFIG_ZMK_RGB_UNDERGLOW_HUE_STEP=10
CONFIG_ZMK_RGB_UNDERGLOW_SAT_STEP=10
CONFIG_ZMK_RGB_UNDERGLOW_BRT_STEP=10
CONFIG_ZMK_RGB_UNDERGLOW_HUE_START=0
CONFIG_ZMK_RGB_UNDERGLOW_SAT_START=100
CONFIG_ZMK_RGB_UNDERGLOW_BRT_START=100
CONFIG_ZMK_RGB_UNDERGLOW_SPD_START=3
CONFIG_ZMK_RGB_UNDERGLOW_EFF_START=0
CONFIG_ZMK_RGB_UNDERGLOW_ON_START=y
# CONFIG_ZMK_RGB_UNDERGLOW_AUTO_OFF_IDLE is not set
# CONFIG_ZMK_RGB_UNDERGLOW_AUTO_OFF_USB is not set
CONFIG_WS2812_STRIP=y
CONFIG_WS2812_STRIP_SPI=y

BTW, it doesn't matter if app/boards/shields/murphpad/boards/nice_nano.conf and app/boards/shields/murphpad/murphpad.conf are consolidated; the results are the same.

romones commented 2 years ago

I think the overrides worked for me when I added them under the boards/shields/keyboardname and not just at the root of the zmk-config. The downside is I had to drag clone all the other files. Example for avalanche keyboard https://github.com/romones/zmk-config/tree/main/config/boards/shields/avalanche
and also works on cradio https://github.com/romones/zmk-config/tree/main/config/boards/shields/avalanche

caksoylar commented 2 years ago

That's expected since you are now building a different shield by renaming things. In that sense you are not overriding anything since ZMK itself doesn't have avalance_left and avalanche_right shield definitions.

romones commented 2 years ago

Well it works for the sweep too. Just wanted to leave a trace here that it's not impossible to override just one half of a split.

Edit: And I realised I completely screwed up my links in the previous comment: Here are all the atrocities I did to the sweep https://github.com/romones/zmk-config-sweep

caksoylar commented 2 years ago

That's good to know. In the processing order of the overlays, the one from the config folder indeed comes later than ZMK's: https://github.com/romones/zmk-config-sweep/runs/8155910697?check_suite_focus=true#step:9:27

caksoylar commented 1 year ago

I wonder if this would be fixed by https://github.com/zmkfirmware/zephyr/pull/24 getting merged, probably worth testing.

caksoylar commented 11 months ago

I'll close this assuming it is fixed as above, please re-open (or someone else comment) if you can still reproduce.

lesshonor commented 3 months ago

This problem is still reproducible, and the issue should be re-opened.

At present, the only way to override overlays that are in ZMK main is to place the necessary changes in your keymap.

If you need to make changes to only one part of a split keyboard, create separate keymaps for each part.

joschkarick commented 2 months ago

I'm having the same issue with my Sofle and nice!nano V2 overlay. The only workaround was the legacy way (config/boards) or placing the config inside the keymap.