I have the problem with PyVirtualDisplay==0.2.5 package for running tests with usage of virtual displays (headless mode)

[ratmirasanov]

My tests are working and running as expected with PyVirtualDisplay==0.2.1, but I have a problem with the latest version of this package (for example, PyVirtualDisplay==0.2.5).

My code snippet:

...
def _set_up(self):

    # Creation of instance of the browser.
    self.display = Display(visible=0, size=(config.WIDTH, config.HEIGHT))
    self.display.start()
    desired = self.get_desired_capabilities(config.BROWSER)
    self.driver = webdriver.Remote(command_executor=config.ACTIVE_NODE, desired_capabilities=desired)
    # Maximize window size.
    self.driver.set_window_size(config.WIDTH, config.HEIGHT)
...

Error message:

Error Traceback (most recent call last): File "/usr/local/Cellar/python/3.7.6/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 59, in testPartExecutor yield File "/usr/local/Cellar/python/3.7.6/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 624, in run self.setUp() File "/Users/ratmirasanov/Documents/testing/welcome_widget/test_welcome_widget.py", line 24, in setUp self._set_up() File "/Users/ratmirasanov/Documents/testing/utilities.py", line 41, in _set_up self.display.start() File "/Users/ratmirasanov/Documents/testing/venv/lib/python3.7/site-packages/pyvirtualdisplay/abstractdisplay.py", line 171, in start raise XStartTimeoutError(msg % d) pyvirtualdisplay.abstractdisplay.XStartTimeoutError: Failed to start X on display ":1001" (xdpyinfo check failed).

I did not find any working solution to fix this problem. Any help will be appreciated. Thanks.

UPD: The PyVirtualDisplay==0.2.5 package is working as expected on Ubuntu 18.04.3 LTS.

Where is located the xdpyindo program: /usr/bin/xdpyinfo.

The problem is on my Mac (macOS Catalina 10.15.2) with XQuartz (https://www.xquartz.org/index.html) installed (xdpyindo program is located in /opt/X11/bin/xdpyinfo folder).

[ponty]

xdpyinfo check was added in #25

Enable logging:

import logging
FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
logging.basicConfig(level=logging.DEBUG, format=FORMAT)

What happens if you delete xdpyinfo?

[ratmirasanov]

I after enabling logging I see the following:

2020-01-06 18:47:56,169 - easyprocess - DEBUG - command: ['Xvfb', '-help'] 2020-01-06 18:47:56,257 - easyprocess - DEBUG - process was started (pid=33990) 2020-01-06 18:47:56,912 - easyprocess - DEBUG - process has ended, return code=0 2020-01-06 18:47:56,913 - easyprocess - DEBUG - command: ['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '1366x768x24', ':1000'] 2020-01-06 18:47:56,913 - easyprocess - DEBUG - command: ['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '1366x768x24', ':1001'] 2020-01-06 18:47:56,917 - easyprocess - DEBUG - process was started (pid=33991) 2020-01-06 18:47:56,918 - pyvirtualdisplay.abstractdisplay - DEBUG - DISPLAY=:1001 2020-01-06 18:47:56,919 - easyprocess - DEBUG - command: ['xdpyinfo'] 2020-01-06 18:47:57,002 - easyprocess - DEBUG - process was started (pid=33992) 2020-01-06 18:47:57,948 - easyprocess - DEBUG - process has ended, return code=1

When I removed xdpyinfo program I have the following in logs and my script is working:

2020-01-06 18:53:18,263 - easyprocess - DEBUG - command: ['Xvfb', '-help'] 2020-01-06 18:53:18,272 - easyprocess - DEBUG - process was started (pid=34303) 2020-01-06 18:53:18,294 - easyprocess - DEBUG - process has ended, return code=0 2020-01-06 18:53:18,295 - easyprocess - DEBUG - command: ['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '1366x768x24', ':1000'] 2020-01-06 18:53:18,296 - easyprocess - DEBUG - command: ['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '1366x768x24', ':1001'] 2020-01-06 18:53:18,300 - easyprocess - DEBUG - process was started (pid=34304) 2020-01-06 18:53:18,302 - pyvirtualdisplay.abstractdisplay - DEBUG - DISPLAY=:1001 2020-01-06 18:53:18,304 - easyprocess - DEBUG - command: ['xdpyinfo'] 2020-01-06 18:53:18,307 - easyprocess - DEBUG - OSError exception: [Errno 2] No such file or directory: 'xdpyinfo': 'xdpyinfo' 2020-01-06 18:53:18,308 - pyvirtualdisplay.abstractdisplay - WARNING - xdpyinfo was not found, X start can not be checked! Please install xdpyinfo! 2020-01-06 18:53:18,417 - selenium.webdriver.remote.remote_connection - DEBUG - POST http://10.51.161.41:1111/wd/hub/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "platformName": "any", "goog:chromeOptions": {"prefs": {"profile.default_content_setting_values.notifications": 1}, "extensions": [], "args": ["--use-fake-ui-for-media-stream", "--use-fake-device-for-media-stream", "--no-sandbox", "--disable-dev-shm-usage"]}}}, "desiredCapabilities": {"browserName": "chrome", "version": "", "platform": "ANY", "loggingPrefs": {"browser": "ALL"}, "goog:chromeOptions": {"prefs": {"profile.default_content_setting_values.notifications": 1}, "extensions": [], "args": ["--use-fake-ui-for-media-stream", "--use-fake-device-for-media-stream", "--no-sandbox", "--disable-dev-shm-usage"]}}} 2020-01-06 18:53:18,433 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): 10.51.161.41:1111 2020-01-06 18:53:20,106 - urllib3.connectionpool - DEBUG - http://10.51.161.41:1111 "POST /wd/hub/session HTTP/1.1" 200 1017 2020-01-06 18:53:20,107 - selenium.webdriver.remote.remote_connection - DEBUG - Finished Request

@ponty, as I understood the xdpyinfo not working?

[ponty]

I added logging for xdpyinfo, please check with it.

[ratmirasanov]

I see the following now:

Error Traceback (most recent call last): File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 59, in testPartExecutor yield File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/case.py", line 624, in run self.setUp() File "/Users/ratmirasanov/Documents/testing/welcome_widget/test_welcome_widget.py", line 24, in setUp self._set_up() File "/Users/ratmirasanov/Documents/testing/utilities.py", line 43, in _set_up self.display.start() File "/Users/ratmirasanov/Documents/testing/venv/lib/python3.7/site-packages/pyvirtualdisplay/abstractdisplay.py", line 171, in start raise XStartTimeoutError(msg % (d, xdpyinfo.stderr)) pyvirtualdisplay.abstractdisplay.XStartTimeoutError: Failed to start X on display ":1001" (xdpyinfo check failed, stderr:[xdpyinfo: unable to open display ":1001".]).

[ponty]

I have no idea. This happens inside PyVirtualDisplay:

$ (Xvfb -br -nolisten tcp -screen 0 1366x768x24 :1001 &);sleep 0.1;DISPLAY=:1001 xdpyinfo 
name of display:    :1001
version number:    11.0
vendor string:    The X.Org Foundation
vendor release number:    12004000
X.Org version: 1.20.4
maximum request size:  16777212 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
number of supported pixmap formats:    6
...

[ratmirasanov]

Ok, thanks anyway for your help. We have such a problem on any Mac (tested on several machines with XQuartz). For now, we will use the old version of the package for Mac and the latest for our Linux machine. Or we can just remove xdpyinfo program on Mac (I think, it's better).

[ponty]

I installed XQuartz on OSX:

$ brew cask install xquartz
$ brew cask info xquartz 
xquartz: 2.7.11 (auto_updates)
https://www.xquartz.org/
/usr/local/Caskroom/xquartz/2.7.11 (74.6MB)
From: https://github.com/Homebrew/homebrew-cask/blob/master/Casks/xquartz.rb
==> Name
XQuartz
==> Artifacts
XQuartz.pkg (Pkg)

$ PATH="$PATH:/opt/X11/bin"

I checked xdpyinfo, no problem:

$ xdpyinfo -version
xdpyinfo 1.3.2

$ (Xvfb -br -nolisten tcp -screen 0 1366x768x24 :1001 &);sleep 0.1;DISPLAY=:1001 xdpyinfo 
name of display:    :1001
version number:    11.0
vendor string:    The X.Org Foundation
vendor release number:    11804000
X.Org version: 1.18.4
maximum request size:  16777212 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
number of supported pixmap formats:    6
...

I checked PyVirtualDisplay, no problem:

# run.py
from pyvirtualdisplay import Display
import logging
FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
logging.basicConfig(level=logging.DEBUG, format=FORMAT)

display = Display(visible=0, size=(300, 200))
display.start()

$ python3 run.py
2020-01-26 09:51:13,022 - easyprocess - DEBUG - command: ['Xvfb', '-help']
2020-01-26 09:51:13,031 - easyprocess - DEBUG - process was started (pid=607)
2020-01-26 09:51:13,036 - easyprocess - DEBUG - process has ended, return code=0
2020-01-26 09:51:13,037 - easyprocess - DEBUG - command: ['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '300x200x24', ':1000']
2020-01-26 09:51:13,038 - easyprocess - DEBUG - command: ['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '300x200x24', ':1001']
2020-01-26 09:51:13,042 - easyprocess - DEBUG - process was started (pid=608)
2020-01-26 09:51:13,043 - pyvirtualdisplay.abstractdisplay - DEBUG - DISPLAY=:1001
2020-01-26 09:51:13,043 - easyprocess - DEBUG - command: ['xdpyinfo']
2020-01-26 09:51:13,079 - easyprocess - DEBUG - process was started (pid=609)
2020-01-26 09:51:13,138 - easyprocess - DEBUG - process has ended, return code=0
2020-01-26 09:51:13,138 - pyvirtualdisplay.abstractdisplay - INFO - Successfully started X with display ":1001".

Can you test the above commands?

[ratmirasanov]

@ponty, after running the following command: (Xvfb -br -nolisten tcp -screen 0 1366x768x24 :1001 &);sleep 0.1;DISPLAY=:1001 xdpyinfo I see:

_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created. _XSERVTransSocketUNIXCreateListener: mkdir(/tmp/.X11-unix) failed, errno = 2 _XSERVTransMakeAllCOTSServerListeners: failed to create listener for local (EE) Fatal server error: (EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE) xdpyinfo: unable to open display ":1001".

[ponty]

Xvfb doesn't start. This is a permission problem. I had the same on TravisCI. I found this solution: http://blog.tigerteufel.de/?p=476

mkdir /tmp/.X11-unix
sudo chmod 1777 /tmp/.X11-unix
sudo chown root /tmp/.X11-unix/

[ratmirasanov]

Yes, my problem is solved. Thanks!!

[Lakshmiaddepalli]

Nice worked for me too!

[mutiev]

Yes, problem solved for macos xvfb problem. Full info

brew install --cask xquartz
export PATH=$PATH:/opt/X11/bin >> .zshrc

mkdir /tmp/.X11-unix
sudo chmod 1777 /tmp/.X11-unix
sudo chown root /tmp/.X11-unix/