Provide means to test ALL existing and future LAYOUTs using actual keymaps.

[mechmerlin]

Background

The advent of the QMK Configurator has made it more apparent that sensible LAYOUTs need to be made. There has been a push to make this happen and the QMK Configurator is a lot more user friendly because of it.

Problem

Unfortunately while these LAYOUTs are used often by QMK Configurator, it's not often used by keymaps in keyboards/some_keeb/keymaps. As a result, a majority of these LAYOUTs are untested until they get to the QMK Configurator stage. This is ONLY true if a user chooses that layout. This results in some LAYOUTs not compiling such as what occured with Pull Request #3542.

Proposal

Create a test directory in keymaps such as

keyboards/your_keeb/keymaps/test

Create a keymap.c that utilizes all available LAYOUTs. Using the minivan as an example, we can do this

const uint16_t PROGMEM keymaps[][MATRIX_ROWS][MATRIX_COLS] = {
  [_QW1] = LAYOUT( /* Qwerty */
    KC_TAB,  KC_Q,    KC_W,    KC_E,    KC_R,    KC_T,    KC_Y,    KC_U,    KC_I,    KC_O,    KC_P,    KC_BSPC,
    MO(_L1), KC_A,    KC_S,    KC_D,    KC_F,    KC_G,    KC_H,    KC_J,    KC_K,    KC_L,    KC_SCLN, MO(_L1),
    KC_LSFT, KC_Z,    KC_X,    KC_C,    KC_V,    KC_B,    KC_N,    KC_M,    KC_COMM, KC_DOT,  KC_SLSH, MO(_L2),
    KC_LCTL, MO(_L2), KC_LGUI,                   KC_ENT,  KC_SPC,                    KC_RALT, KC_ESC,  TG(_L3)
  ),
  [_QW2] = LAYOUT_arrow_command( /* Qwerty */
    L2_ESC,  KC_Q,    KC_W,    KC_E,    KC_R,    KC_T,    KC_Y,    KC_U,    KC_I,    KC_O,    KC_P,    KC_BSPC,
    L1_TAB,  KC_A,    KC_S,    KC_D,    KC_F,    KC_G,    KC_H,    KC_J,    KC_K,    KC_L,    KC_SCLN, KC_ENT,
    KC_LSFT, KC_Z,    KC_X,    KC_C,    KC_V,    KC_B,    KC_N,    KC_M,    KC_COMM, KC_DOT,  KC_UP,   L2_SLSH,
    TD_LCTL, MO(_L3), TD_LALT, TD_LGUI,          KC_SPC,  KC_SPC,           MO(_L1), KC_LEFT, KC_DOWN, KC_RGHT
    ),

  [_QW3] = LAYOUT_arrow(
    KC_TAB, KC_Q,   KC_W,   KC_E, KC_R,   KC_T,   KC_Y,   KC_U, KC_I,   KC_O,   KC_P,   KC_BSPC,
    KC_LCTL,KC_A,   KC_S,   KC_D, KC_F,   KC_G,   KC_H,   KC_J, KC_K,   KC_L,   KC_SCLN,KC_ENT,
    KC_LSFT,KC_Z,   KC_X,   KC_C, KC_V,   KC_B,   KC_N,   KC_M, KC_COMM,KC_DOT, KC_SLSH,KC_RSFT,
    FTN,    KC_LGUI,KC_LALT,              LOWER,  RAISE,        KC_QUOT,KC_LBRC,KC_RBRC,KC_RALT
  )
};

You may have noticed that it is not complete, as not all the minivan's LAYOUTs are used by any of the code in tv44/keymaps.

While the resulting hex would not work very well on any keyboard, this will atleast allow us to

1. Check for compilation in Travis.
2. Ensure that changes to existing LAYOUTs will be tested

People adding new LAYOUTs will also have to be vigilant to also add on to this.

At over 155 boards in qmk_firmware and with each board supporting several LAYOUT macros. This may take awhile to implement.

[skullydazed]

I think we have a better solution for this problem already started, with some polishing work still to do. The API is being continuously exercised now by a process that attempts to compile keymaps for every keyboard/layout. The beta page showing this is here:

https://yanfali.github.io/qmk_error_page/

Next step is to expose compiler output, which I've exposed on the API but that information hasn't been folded into the error page yet.

[mechmerlin]

Is there a way so that when a developer is adding a new LAYOUT, they have a way to check that their new LAYOUT can be compiled without submitting a PR, merging, and then checking that site?

[jbtompkins]

I'd also be interested in a testing scheme that doesn't require your layout to be merged.

[drashna]

@jbtompkins Actually, we already support that. Set up Travis CI on your fork, and push to your fork. The already included Travis scripts should run a number of tests.

I'm not sure if this link will work, but, for instance: https://travis-ci.org/drashna/qmk_firmware/branches

Also, many of the scripts may work on a local system, as well.

@mechmerlin Yes, set up Travis CI to work on your local fork. Maybe we need a page in the docs on how to set that up, and use it.

[jackhumbert]

We have the qmk_web_stack now that may make these a bit easier - @mechmerlin do you think this addresses that sort of use case?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 90 days. It will be closed in the next 30 days unless it is tagged properly or other activity occurs. For maintainers: Please label with bug, in progress, on hold, discussion or to do to prevent the issue from being re-flagged.

[tzarc]

Closing due to inactivity.