[PLAN] Key logger for data collection

[xjjak]

We need a keylogger for our keyboards. This would enable easier collection of larger datasets, which would also be less unbalanced and more realistic.

Possible approaches:

We tried using the keycodes sent by the keyboard, making it completely device-side. This was not very successful. It should be attempted to utilize the QMK firmware, possibly with custom code snippets to make collection easier or even on the keyboards themselves. This could be done by sending key numbers through a separate channel, similar to something described in this article under Firmware.

[xjjak]

To add key logging output to my keyboard with the following additions/modifications on to of the tutorial:

• Follow basic qmk instructions for building and flashing
• locate the folder with keymap.json/keymap.c
• If there is only keymap.json: run qmk json2c keymap.json > keymap.c
• remove/rename the .json
• change the bootloader in rules.mk to atmel-dfu
• Append the following to keymap.c:

  bool process_record_user(uint16_t keycode, keyrecord_t *record) {
    #ifdef CONSOLE_ENABLE
        const bool is_combo = IS_COMBOEVENT(record->event); // DIFFERS FROM TUTORIAL
        uprintf("0x%04X,%u,%u,%u,%b,0x%02X,0x%02X,%u\n",
             keycode,
             is_combo ? 254 : record->event.key.row,
             is_combo ? 254 : record->event.key.col,
             get_highest_layer(layer_state),
             record->event.pressed,
             get_mods(),
             get_oneshot_mods(),
             record->tap.count
             );
    #endif
    // switch (keycode) {
    // //...
    // }
    return true;
  }

  The line is_combo check had to be changed as COMBO_EVENT wasn't found. There is a wrapper function available called IS_COMBOEVENT which returns a bool.

• Now try compilation with qmk compile (assuming standard kb and km are set) I encountered the following error:

  Compiling: platforms/avr/drivers/i2c_master.c                                                      In function 'i2c_start_impl',
    inlined from 'i2c_start' at platforms/avr/drivers/i2c_master.c:104:18:
  platforms/avr/drivers/i2c_master.c:61:10: error: array subscript 0 is outside array bounds of 'volatile uint8_t[0]' {aka 'volatile unsigned char[]'} [-Werror=array-bounds=]
   61 |     TWCR = 0;
      |          ^

  Reddit led me here with a workaround to be found here

  The workaround disables this type of error, by appending -e AVR_CFLAGS="-Wno-array-bounds" to qmk compile or qmk flash

[xjjak]

The plan now is to write a simple script the output of hid_listen can be piped into, which prepends the current Unix timestamp. A format changing "filter" can be applied to the whole file afterward if necessary for compatibility with the current training method.

[xjjak]

For the HID console, the QMK udev rules from qmk_fimware/utils/udev/50-qmk.rules must be copied to lib/udev/rules.d or that directory in its corresponding location.

[xjjak]

To mitigate the effect that a press event was only sent after a hold-event was detected when a key was kept pressed, the modified function was changed to pre_process_user_record(). Thanks, @palisn, for figuring that out.