send special keys?

[stuart-little]

This is a follow-up on an earlier discussion, where it was suggested it would make for a good feature request.

It would be convenient to be able to send special-key combos to the window (or other elements), like say Ctrl+Shift+k to open the web console.

The reference implementation of the Marionette driver (a Python module) illustrates how to do this: the special keys are assigned codepoints in the Unicode Private Use Area, all of the form U+EXXX (rendered in Python as "\uexxx").

Other sources seem to agree on the encoding used by Firefox for these special keys.

[david-dick]

agreed. planning on an implementation within 24 hours.

[stuart-little]

Thanks! I'm curious, is it possible to do this now, through some unicode magic? I tried the rather naive

#!/usr/bin/env perl
use warnings;
use v5.12;

use utf8;
use Firefox::Marionette;

my $firefox = Firefox::Marionette->new(
    visible => 1,
    );

$firefox->context('chrome');
$firefox->await(sub { $firefox->interactive() && $firefox->find_id('main-window') })->type("\N{U+e00a}");

sleep;

, hoping to send the Alt key (which would be visible as opening the top menu on the window), only to have it die with

Wide character in syswrite at $HOME/.perlbrew/libs/perl-5.33.9@533main/lib/perl5/Firefox/Marionette.pm line 6933.

[david-dick]

No idea if the marionette method ElementSendKeys method will allow a single special key. Unfortunately, even if that is allowed, the current json encoding is translating to UTF-8 instead of the marionette acceptable \uxxxx type encoding. Marionette uses the PerformActions and ReleaseActions methods to allow pressing and releasing keys in a sequence, which you can see in the documentation is not implemented. That will corrected with the incoming patch.

[stuart-little]

No idea if the marionette method ElementSendKeys method will allow a single special key.

Ah, that part should be fine: in the Selenium Python package

#!/usr/bin/env python

from selenium.webdriver import Firefox, DesiredCapabilities, FirefoxProfile
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

import time

webdriver = Firefox()
webdriver.get("about:blank")

try:
    with webdriver.context(webdriver.CONTEXT_CHROME):
        time.sleep(3)        
        win = webdriver.find_element(By.ID, "main-window")
        win.send_keys(Keys.ALT)

except Exception as error:
   pass

while(1):
    time.sleep(0.1)

works exactly as expected: opens a blank page, and after a few seconds drops down the top menu because of that Alt-key event.

Unfortunately, even if that is allowed, the current json encoding is translating to UTF-8 instead of the marionette acceptable \uxxxx type encoding. Marionette uses the PerformActions and ReleaseActions methods to allow pressing and releasing keys in a sequence, which you can see in the documentation is not implemented. That will corrected with the incoming patch.

I see; will be looking forward to that then. Thanks!

[david-dick]

The equivalent command post patch would be;

Firefox::Marionette->new( visible => 1 )->go("about:blank")->chrome()->find_id("main-window")->type(ALT());

which does work. Who knew?

[david-dick]

okay. pushed to github. feel free to re-open if this patch doesn't work for you.

[stuart-little]

Thank you! I'll test and get back if I find anything amiss, but a quick question: are these typos?

~/repos/firefox-marionette$ ack -Q '(CONTROL()' .

t/01-marionette.t
2036:   ok(CONTROL() eq chr 0xE009, "CONTROL() is correct as OxE009 (Same as CONTROL_LEFT())");

README.md
822:                                 $firefox->key_down(CONTROL(),
836:                                 $firefox->key_down(CONTROL(),
840:                                 $firefox->key_up(CONTROL()
1013:                                 $firefox->key_down(CONTROL(),
1016:                                 $firefox->key_up(CONTROL()
1068:                                 $firefox->key_down(CONTROL(),

lib/Firefox/Marionette/Keys.pm
122:                                 $firefox->key_down(CONTROL(),
125:                                 $firefox->key_up(CONTROL()

lib/Firefox/Marionette.pm
8082:                                 $firefox->key_down(CONTROL(),
8096:                                 $firefox->key_down(CONTROL(),
8100:                                 $firefox->key_up(CONTROL()
8321:                                 $firefox->key_down(CONTROL(),
8324:                                 $firefox->key_up(CONTROL()
8376:                                 $firefox->key_down(CONTROL(),

README
1049:                                     $firefox->key_down(CONTROL(),
1064:                                     $firefox->key_down(CONTROL(),
1068:                                     $firefox->key_up(CONTROL()
1399:                                     $firefox->key_down(CONTROL(),
1402:                                     $firefox->key_up(CONTROL()
1464:                                     $firefox->key_down(CONTROL(),

Except for the first one, all of the other lines seem to be missing a closing ) after CONTROL().

[david-dick]

good catch. fixes pushed.

[stuart-little]

OK, I was finally able to sit in front of a computer and look at the thing: yes, it works; thank you! Here it is, two ways.

• First, as per the docs,

#!/usr/bin/env perl
use warnings;
use v5.12;

use Firefox::Marionette;
use Firefox::Marionette::Keys qw(:all);

my $firefox = Firefox::Marionette->new(
    visible => 1,
    );

$firefox->chrome()->perform(
    $firefox->key_down(ALT()),
    $firefox->key_up(ALT()),
    )->content();

sleep;

opens the menu.

• Secondly though, this now also works:

  
  #!/usr/bin/env perl
  use warnings;
  use v5.12;

use Firefox::Marionette;

my $firefox = Firefox::Marionette->new( visible => 1, );

$firefox->chrome()->await(sub { $firefox->interactive() && $firefox->find_id('main-window') })->type("\N{U+e00a}");

sleep;

as does the version with `ALT()` in place of `"\N{U+e00a}"` (as you [wrote](https://github.com/david-dick/firefox-marionette/issues/9#issuecomment-841745611)), if I add 

use Firefox::Marionette::Keys qw(:all);

to the preamble. 

So all good!