Letter key presses not detected and quick presses in succession not detected - Edge + key press detection

[jwzimmer-zz]

Potential new issues being reported with Edge + keyboard detection, reported via https://scratch.mit.edu/discuss/topic/251930/ (the more recent pages/ posts):

I've run the keyboard test project below and it seems there are two problems when using Edge:
1. The bug where letter keys do not trigger as being pressed.
2. Another bug where pressing and holding a key and then immediately pressing a different key causes the second key to not be recognized.  Here's the repro case:
  - Press and hold the left arrow button for 10 seconds.  The left arrow button is shown as being pressed
  - Release the left arrow button and immediately press the right arrow button.
EXPECTED:  The left arrow shows as being not-pressed and the right arrow shows as being pressed.
OBSERVED: The left arrow continues to show being pressed for another 2-4 seconds after its key-up, and the right arrow doesn't show being pressed at all.
Can anyone else confirm this result?

@cwillisf said this could be a bug / is worth trying to repro, "probably introduced in v457. If the latest Edge is fixed then maybe we should consider removing the keyboard change I made in v457..."

[jwzimmer-zz]

Repro: I repro'd an issue where pressing keys in quick succession does not always cause the 2nd key press to be detected. It was pretty easy to repro in Edge 40, but only repro'able in Edge 41 with very quick 2nd key presses - it's pretty hard to repro in Edge 41.

However, repro is unreliable since it was on a VM, from discussion with @cwillisf:

I'm messing around with this in various browsers but as far as what should happen, I think it's reasonable to expect "wait until key pressed" to catch a quick press. It's theoretically less reliable than a "when key pressed" hat since it's only sampling once per "tick" instead of watching for the key events, but I wouldn't expect the difference to be noticeable except on a very slow computer / very complex project. That is, situations where the frame rate / tick rate is slowed enough that a quick keypress could happen completely in between ticks.

_Edge 40.15063.0.0, Win 10 VM (Win10preview)

• The key presses in project 157805972 are working for me.
• Looking at 20966625:
  • I see the apostrophe triggered when right arrow key is pressed as reported in the thread, but based on TheLogFather's explanation that seems to be due to the nature of hacked key detection blocks rather than the key press detection itself.
  • All non-alphanumeric keys besides the arrow keys are not detected; for example, - and = are not detected
  • Combinations of up 2, 3, 4 arrow keys are detected correctly
  • If I press keys too quickly in succession, the second key press will not register
  • Combinations of 2 - 8 alphanumeric keys are detected correctly (I didn't try more than 8 at once)
• I can repro issue no. 2 described in the thread, using the repro steps they provided.
• I cannot repro issue no. 1 outside of the context where I press one key immediately after another (or itself), and outside of non-alphanumeric, non-arrow keys being detected
  • I believe the keys failing to be detected are only detected through hacked blocks/ not detected by the default "key pressed" block, and that we knew that would happen with the fix to other key press detection (https://github.com/LLK/scratch-flash/pull/1345#issuecomment-323876562)?

Issue no. 2

2. Another bug where pressing and holding a key and then immediately pressing a different key causes the second key to not be recognized.  Here's the repro case:
  - Press and hold the left arrow button for 10 seconds.  The left arrow button is shown as being pressed
  - Release the left arrow button and immediately press the right arrow button.
EXPECTED:  The left arrow shows as being not-pressed and the right arrow shows as being pressed.
OBSERVED: The left arrow continues to show being pressed for another 2-4 seconds after its key-up, and the right arrow doesn't show being pressed at all.

Edge 41.16299.15.0, Win 10 VM (Win 10-VMWare)

• The same set of keys as described above register presses
• Combinations of keys register presses (as described above)
• Same experience described above re repro'ing issue no. 1 & 2 except that I only saw it a few times; the issue seems to be significantly harder to repro and only occurs if I press & release the key very quickly... maybe too quickly to be reasonable?

There is also sometimes a lag in when the key press stops being detected if you hold down a key or when you see the 2nd key press appear on screen after you press 2 keys, but this could be due to using a VM. I see the lag when typing in the Scratch search bar as well, so I don't think it is due to the key detection - I think it is likely the VM.

(The quickest key presses I can make do register when I type in the Scratch search bar, so I don't think they're too fast for anything to detect... I'm not sure if we expect the key pressed sensing block to detect them, though.)

[cwillisf]

I've heard suggestions that Edge has since fixed the bug that caused #1334, and since #1341 & #1345 are causing their own issues (like this and #1363) I think it might be best to look into reverting them.

[jwzimmer-zz]

Trying to repro on a base OS rather than a VM

Not repro'd in Edge 41 @cwillisf was not able to repro the issue using Edge 41 on his Windows computer.

Repro'd in Edge 40, apparently I re-booted the QA machine into Windows 10 and let it install updates; the version of Edge at that point was 40.15063.0.0. I am able to repro the issue where a second, quick key press doesn't always register using project 20966625 (which specifically uses the key pressed sensing block). However, it is not consistent - I have to try it several times to encounter the issue. The repro steps are as described above.

I used 180767378 to see if I could see the "missed" presses, and based on what I could observe in the changes of the say/ think bubbles, "when key pressed" seemed to work correctly, but with "key pressed", "wait until key pressed", and "if key pressed then", it did appear that sometimes a second quick press might not register. However, from talking to @cwillisf, the way the editor renders the stage and the say/ think bubbles in particular complicate whether what I can see accurately reflects whether the key detection is working correctly. (@cwillisf said "when turbo mode is off, loops in Scratch automatically wait until the next graphical update before looping back to the top, and also both the "say" and "think" commands wait for a graphical update too, so you're guaranteed to see at least one frame of each message".)

In turbo mode, I think I can repro issue 2 (secondary key press not always detected) in project 20966625 (Edge 40). It is difficult though - I had to try it quite a few times before I thought I saw it, and I might have just missed it (relying on the visual impact of the key press isn't fool proof, for numerous reasons).

(Still in turbo mode) I did notice that after many presses you could build up a backlog of key presses that would continue to be executed on stage after you stopped pressing the keyboard altogether. This is easy to repro. @cwillisf said:

if there's a limit to that buffer that could be the whole problem

To repro the backlog of key presses:

• Open project 20966625
• Shift + click green flag to toggle into Turbo Mode
• Click green flag
• Press many keys quickly in quick succession, e.g. "ghgghghghghghghghghghg" then add different keys e.g. "gkgkgkgkgkgkgjfjfjfjfjfjdudududududjsskssjsksk"
  • You will notice that the keys being highlighted on screen are out of sync with what you are currently typing
• Stop touching the keyboard
• Note that key presses still continue to be executed on the project stage