Open yannrichet opened 4 years ago
python3 -c "import pynput; pynput.keyboard.Controller().press(pynput.keyboard.Key.space)" works well, while python3 -c "import pynput; pynput.keyboard.Controller().press(pynput.keyboard.Key('a'))" does not give anything...
the second line is working if you use the KeyCode.from_char
class instead:
python3 -c "import pynput; pynput.keyboard.Controller().press(pynput.keyboard.KeyCode.from_char('a'))"
but you can also use the plain character as argument to the press
method:
python3 -c "import pynput; pynput.keyboard.Controller().press('a')"
Right, my mistake, sorry. But it remains inactive inside my x11vnc/docker...
I am afraid that I do not completely understand your use case: are your trying to control x11vnc
from your host using pynput, or are you running pynput on the remote machine?
If you provide a script to reproduce the issue, troubleshooting would be greatly simplified.
Yes, you are right :) I try to use pynput on the remote machine (in facts, it is not a 'true' remote one, just a docker one). Here is the needed stuff & process (remove all .txt extensions):
sudo docker build -t snitch .
sudo ./snitch-docker.sh
. It will start the docker and everything, so you can just paste python3 -c "import pynput; pynput.keyboard.Controller().press('a')"
in the xterm... and nothing happens...while the expression was wrong, it seems that I was wrong too : it doesn't work with Xvfb. Here's a minimal example exhibiting the issue, with the following Dockerfile:
FROM ubuntu:18.04
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y xvfb python3-tk python3-dev python3-pip
RUN python3 -m pip install pynput
ENV DISPLAY=:0.0
reproduce with:
$ docker build -t pynput .
$ docker run -it --rm pynput bash
# Xvfb :0.0 -screen 0 1600x900x24 &
# python3 -c "import pynput; pynput.keyboard.Controller().type('XXX')"
nothing is printed to the console.
might be related to https://github.com/python-xlib/python-xlib/issues/156 ?
while the expression was wrong, it seems that I was wrong too : it doesn't work with Xvfb. Here's a minimal example exhibiting the issue, with the following Dockerfile:
FROM ubuntu:18.04 RUN apt-get update RUN DEBIAN_FRONTEND=noninteractive apt-get install -y xvfb python3-tk python3-dev python3-pip RUN python3 -m pip install pynput
reproduce with:
$ docker build -t pynput . $ docker run -it --rm pynput bash # Xvfb :0.0 -screen 0 1600x900x24 & # python3 -c "import pynput; pynput.keyboard.Controller().type('XXX')"
nothing is printed to the console.
Well, I received 'Xlib.error.DisplayNameError: Bad display name ""' when running python3 -c "import pynput; pynput.keyboard.Controller().type('XXX')"
. But this is solved using just export DISPLAY=:0
:
root@085be1766e78:/# Xvfb :0.0 -screen 0 1600x900x24 &
[1] 11
root@085be1766e78:/# python3 -c "import pynput; pynput.keyboard.Controller().type('XXX')"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.6/dist-packages/pynput/__init__.py", line 40, in <module>
from . import keyboard
File "/usr/local/lib/python3.6/dist-packages/pynput/keyboard/__init__.py", line 52, in <module>
from ._xorg import KeyCode, Key, Controller, Listener
File "/usr/local/lib/python3.6/dist-packages/pynput/keyboard/_xorg.py", line 39, in <module>
from pynput._util.xorg import (
File "/usr/local/lib/python3.6/dist-packages/pynput/_util/xorg.py", line 40, in <module>
_check()
File "/usr/local/lib/python3.6/dist-packages/pynput/_util/xorg.py", line 38, in _check
display = Xlib.display.Display()
File "/usr/local/lib/python3.6/dist-packages/Xlib/display.py", line 89, in __init__
self.display = _BaseDisplay(display)
File "/usr/local/lib/python3.6/dist-packages/Xlib/display.py", line 71, in __init__
protocol_display.Display.__init__(self, *args, **keys)
File "/usr/local/lib/python3.6/dist-packages/Xlib/protocol/display.py", line 84, in __init__
name, protocol, host, displayno, screenno = connect.get_display(display)
File "/usr/local/lib/python3.6/dist-packages/Xlib/support/connect.py", line 73, in get_display
return mod.get_display(display)
File "/usr/local/lib/python3.6/dist-packages/Xlib/support/unix_connect.py", line 76, in get_display
raise error.DisplayNameError(display)
Xlib.error.DisplayNameError: Bad display name ""
root@085be1766e78:/# $DISPLAY
root@085be1766e78:/# export DISPLAY=:0
root@085be1766e78:/# python3 -c "import pynput; pynput.keyboard.Controller().type('XXX')"
root@085be1766e78:/#
Anyway, 'x11vnc' is never used by you, @gregseth . So it might not be related to...
@yannrichet yeah, fixed the Dockerfile in my comment to set the $DISPLAY variable properly
One more step : the following code works well on docker:
import pynput
import Xlib
k=pynput.keyboard.Controller()
self=k
Xlib.ext.xtest.fake_input(self._display,Xlib.X.KeyPress,self._display.keysym_to_keycode(Xlib.XK.string_to_keysym('a')))
self._display.sync()
which let me think that it should work with fake_input anyway...
Well, I can solve the issue when I force calling fake_input instead of send_event in pynput.keyboard._xorg:
def _handle(self, key, is_press):
...
# If the key has a virtual key code, use that immediately with
# fake_input; fake input,being an X server extension, has access to more
# internal state that we
if key.vk is not None:
with display_manager(self._display) as dm:
Xlib.ext.xtest.fake_input(
dm,
Xlib.X.KeyPress if is_press else Xlib.X.KeyRelease,
dm.keysym_to_keycode(key.vk))
# Otherwise use XSendEvent; we need to use this in the general case to
# work around problems with keyboard layouts
else:
with display_manager(self._display) as dm:
Xlib.ext.xtest.fake_input(
dm,
Xlib.X.KeyPress if is_press else Xlib.X.KeyRelease,
dm.keysym_to_keycode(keysym))
# try:
# keycode, shift_state = self.keyboard_mapping[keysym]
# self._send_key(event, keycode, shift_state)
#
# except KeyError:
# with self._borrow_lock:
# keycode, index, count = self._borrows[keysym]
# self._send_key(
# event,
# keycode,
# index_to_shift(self._display, index))
# count += 1 if is_press else -1
# self._borrows[keysym] = (keycode, index, count)
but I cannot decide how to properly implement this in _send_key()
instead.
@moses-palmer , what do you think is better ? Do we need to detect Xvfb (I imagine there is a way to), and so return to the baseline af using Xlib.ext.xtest.fake_input
?
Thank you for your detailed instructions.
I ran your dockerfile and managed to confirm that it indeed did not work. Using xev
inside the container I found that the only discernible difference between the events sent when using pynput and using the XTEST extension is the SYNTHETIC flag. I guess this is possible since XTEST is an extension that runs inside the server.
I have a partial solution in fixup-xorg-fake-events
. It ensures that keys create from characters---such as pynput.keyboard.KeyCode.from_char('a')
or anything passed to pynput.keyboard.Controller.type
---sets its vk
attribute to the corresponding keysym.
This ensures that the first conditional in your snippet above succeeds, and XTEST is used. I have not run any extensive tests on this branch however, other than ensuring that unmapped keys fail.
I get a new error, which might be just related to the new lines:
# Create a display to verify that we have an X connection
DISPLAY = Xlib.display.Display()
atexit.register(DISPLAY.close)
...
def display_manager(display=DISPLAY):
...
:
Traceback (most recent call last): File "/usr/local/bin/snitch", line 11, in
sys.exit(main()) File "/usr/local/lib/python3.6/dist-packages/snitch/main.py", line 42, in main results = WIN.playback(include_snapshots=True) File "/usr/local/lib/python3.6/dist-packages/snitch/ui/controller.py", line 134, in playback key_event_catcher=catcher File "/usr/local/lib/python3.6/dist-packages/snitch/player.py", line 74, in play event.execute() File "/usr/local/lib/python3.6/dist-packages/snitch/model/data/events.py", line 263, in execute kbd.type(self.text) File "/usr/local/lib/python3.6/dist-packages/pynput/keyboard/_base.py", line 466, in type self.press(key) File "/usr/local/lib/python3.6/dist-packages/pynput/keyboard/_base.py", line 386, in press self._handle(resolved, True) File "/usr/local/lib/python3.6/dist-packages/pynput/keyboard/_xorg.py", line 256, in _handle dm.keysym_to_keycode(key.vk)) File "/usr/lib/python3.6/contextlib.py", line 88, in exit next(self.gen) File "/usr/local/lib/python3.6/dist-packages/pynput/_util/xorg.py", line 76, in display_manager raise X11Error(errors) pynput._util.xorg.X11Error: [(BadValue(<Xlib.display._BaseDisplay object at 0x7fb2c24ba710>, b'\x00\x02\x17\x00\x00\x00\x00\x00\x02\x00\x84\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'), None)]
That it interesting. This feature branch adds error checking where there was none before. Do you think you could provide a listing of the values passed to dm.keysym_to_keycode(key.vk))
on line 256 in pynput/keyboard/_xorg.py
?
Seems strange : recording works well (both keyboard and mouse), but replaying fails (prints some spaces instead of letters). I used the standard examples. Also, using debug mode of x11vnc shows that nothing is given as input to x11vnc...
Regards.