search

keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
368 stars 102 forks source link

feat(web): add recent-history log to gesture engine #11277

Closed jahorton closed 2 weeks ago

jahorton commented 2 weeks ago

To facilitate debugging of the gesture engine when needed, this PR adds configurable history-tracking to the gesture engine.

Within KMW:

// Provides an array of the live-JS form of the most recently-seen gestures
keyman.osk.vkbd.gestureEngine.history

// Provides a prettified JSON version of the latter - great for preserving as text logs
console.log(keyman.osk.vkbd.gestureEngine.historyJSON)

@keymanapp-test-bot skip

keymanapp-test-bot[bot] commented 2 weeks ago

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

jahorton commented 2 weeks ago

Example JSON-log for a quick sil_euro_latin down-flick:

[
  [
    {
      "gestureSetId": "default",
      "matchedId": "flick-start",
      "linkType": "chain",
      "sources": [
        {
          "identifier": "touch:1",
          "isFromTouch": true,
          "path": {
            "coords": [
              {
                "targetX": 105,
                "targetY": 105.4521484375,
                "t": 4361,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 106.32171630859375,
                "t": 4393,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 107.19134521484375,
                "t": 4398.099999964237,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 111.53915405273438,
                "t": 4404.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 115.88699340820312,
                "t": 4413,
                "item": "shift-K_D"
              },
              {
                "targetX": 105.86958312988281,
                "targetY": 121.97393798828125,
                "t": 4420.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 107.60869598388672,
                "targetY": 131.53912353515625,
                "t": 4427.399999976158,
                "item": "shift-K_D"
              }
            ],
            "stats": {
              "angle": 3.0419240590114853,
              "cardinal": "s",
              "netDistance": 26.21708535444175,
              "duration": 66.39999997615814,
              "sampleCount": 7,
              "rawDistance": 26.305590510382302
            }
          },
          "stateToken": "shift"
        }
      ],
      "allSourceIds": [
        "touch:1"
      ]
    },
    {
      "gestureSetId": "default",
      "matchedId": "flick-mid",
      "linkType": "chain",
      "sources": [
        {
          "identifier": "touch:1",
          "isFromTouch": true,
          "path": {
            "coords": [
              {
                "targetX": 105,
                "targetY": 105.4521484375,
                "t": 4361,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 106.32171630859375,
                "t": 4393,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 107.19134521484375,
                "t": 4398.099999964237,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 111.53915405273438,
                "t": 4404.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 115.88699340820312,
                "t": 4413,
                "item": "shift-K_D"
              },
              {
                "targetX": 105.86958312988281,
                "targetY": 121.97393798828125,
                "t": 4420.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 107.60869598388672,
                "targetY": 131.53912353515625,
                "t": 4427.399999976158,
                "item": "shift-K_D"
              }
            ],
            "stats": {
              "angle": 3.0419240590114853,
              "cardinal": "s",
              "netDistance": 26.21708535444175,
              "duration": 66.39999997615814,
              "sampleCount": 7,
              "rawDistance": 26.305590510382302
            }
          },
          "stateToken": "shift"
        }
      ],
      "allSourceIds": [
        "touch:1"
      ]
    },
    {
      "gestureSetId": "default",
      "matchedId": "flick-end",
      "linkType": "complete",
      "item": "shift-K_SPACE",
      "sources": [
        {
          "identifier": "touch:1",
          "isFromTouch": true,
          "path": {
            "coords": [
              {
                "targetX": 105,
                "targetY": 105.4521484375,
                "t": 4361,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 106.32171630859375,
                "t": 4393,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 107.19134521484375,
                "t": 4398.099999964237,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 111.53915405273438,
                "t": 4404.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 105,
                "targetY": 115.88699340820312,
                "t": 4413,
                "item": "shift-K_D"
              },
              {
                "targetX": 105.86958312988281,
                "targetY": 121.97393798828125,
                "t": 4420.5,
                "item": "shift-K_D"
              },
              {
                "targetX": 107.60869598388672,
                "targetY": 131.53912353515625,
                "t": 4427.399999976158,
                "item": "shift-K_D"
              },
              {
                "targetX": 108.47826385498047,
                "targetY": 141.1043701171875,
                "t": 4433.799999952316,
                "item": "shift-K_X"
              },
              {
                "targetX": 109.34783172607422,
                "targetY": 151.53912353515625,
                "t": 4441.5,
                "item": "shift-K_X"
              },
              {
                "targetX": 109.34783172607422,
                "targetY": 161.97393798828125,
                "t": 4448.599999964237,
                "item": "shift-K_X"
              },
              {
                "targetX": 109.34783172607422,
                "targetY": 171.53912353515625,
                "t": 4456.299999952316,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 181.97393798828125,
                "t": 4464.5,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 192.40869140625,
                "t": 4481.199999988079,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 202.843505859375,
                "t": 4486.799999952316,
                "item": "shift-K_X"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 214.1478271484375,
                "t": 4494.399999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.08695220947266,
                "targetY": 217.6260986328125,
                "t": 4503.099999964237,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 223.7130126953125,
                "t": 4511.399999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 223.7130126953125,
                "t": 4512.299999952316,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 227.1912841796875,
                "t": 4517.399999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 230.6695556640625,
                "t": 4526.799999952316,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 111.9565200805664,
                "targetY": 234.1478271484375,
                "t": 4531.699999988079,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 235.886962890625,
                "t": 4538.899999976158,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 237.6260986328125,
                "t": 4545.799999952316,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 238.49566650390625,
                "t": 4555.199999988079,
                "item": "shift-K_SPACE"
              },
              {
                "targetX": 112.82608795166016,
                "targetY": 239.36517333984375,
                "t": 4570.799999952316,
                "item": "shift-K_SPACE"
              }
            ],
            "wasCancelled": false,
            "stats": {
              "angle": 3.083217477723039,
              "cardinal": "s",
              "netDistance": 134.14151442086398,
              "duration": 209.79999995231628,
              "sampleCount": 25,
              "rawDistance": 134.61826311234893
            }
          },
          "stateToken": "shift"
        }
      ],
      "allSourceIds": [
        "touch:1"
      ]
    }
  ]
]
keyman-server commented 2 weeks ago

Changes in this pull request will be available for download in Keyman version 17.0.312-beta