search

ponty / PyVirtualDisplay

Python wrapper for Xvfb, Xephyr and Xvnc
BSD 2-Clause "Simplified" License
697 stars 78 forks source link

XStartTimeoutError since v2.0 #62

Closed The-Compiler closed 10 months ago

The-Compiler commented 3 years ago

Since upgrading to v2.0, I sometimes see this on GitHub Actions:

Wed, 20 Jan 2021 17:02:27 GMT py36-pyqt512 run-test: commands[1] | /home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/bin/python -bb -m pytest tests
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR> Traceback (most recent call last):
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/_pytest/main.py", line 265, in wrap_session
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     config._do_configure()
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/_pytest/config/__init__.py", line 982, in _do_configure
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pluggy/hooks.py", line 308, in call_historic
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     res = self._hookexec(self, self.get_hookimpls(), kwargs)
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     return outcome.get_result()
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     raise ex[1].with_traceback(ex[2])
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     res = hook_impl.function(*args)
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pytest_xvfb.py", line 93, in pytest_configure
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     config.xvfb.start()
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pytest_xvfb.py", line 54, in start
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     self._virtual_display.start()
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pyvirtualdisplay/display.py", line 70, in start
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     self._obj.start()
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pyvirtualdisplay/abstractdisplay.py", line 152, in start
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     self._start1_has_displayfd()
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pyvirtualdisplay/abstractdisplay.py", line 200, in _start1_has_displayfd
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     self.display = int(self._wait_for_pipe_text(rfd))
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>   File "/home/runner/work/qutebrowser/qutebrowser/.tox/py36-pyqt512/lib/python3.6/site-packages/pyvirtualdisplay/abstractdisplay.py", line 295, in _wait_for_pipe_text
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR>     % (self._program, self._command,)
Wed, 20 Jan 2021 17:02:44 GMT INTERNALERROR> pyvirtualdisplay.abstractdisplay.XStartTimeoutError: No reply from program Xvfb. command:['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '800x600x16', '-displayfd', '10']

This is with pytest-xvfb which is a rather small wrapper around PyVirtualDisplay.

I think it only happens in the Ubuntu 18.04 environment, not with 20.04. Also it only happens one out of 10 times or so...

Any idea what could be going on there?

ponty commented 3 years ago

This is a 10 sec timeout in start(), but this was present before v2.0. If the processing is very slow then it can fail. I removed this timeout on master branch. Can you test it?

How can I see the log under https://github.com/The-Compiler/pytest-xvfb/actions? The last entry is 6 months old.

Czaki commented 3 years ago

here is my example of fail https://github.com/4DNucleome/PartSeg/runs/1969237540?check_suite_focus=true

ernyigoh commented 3 years ago

I am getting the same error as well with Ubuntu v18.04 and pyvirtualdisplay v2.0. Any idea? image

ponty commented 3 years ago

I removed the timeout in v2.1. Please test it.

intruder777 commented 3 years ago

I'm afraid removing timeout is not a good option. I have experienced kind of "eternal" hang of my script with v2.2. The host system experienced high load and several scripts hung on _wait_for_pipe_text() function of PyVirtualDisplay. And even when the high load was gone, and system was idling, those scripts still were hanging. So the loop inside _wait_for_pipe_text() was running for more than 24 hours before I manually interrupted the scripts.

Becheler commented 2 years ago

Hello, I have the same error being raised in a Ubuntu 20.04 LTS container:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.8/dist-packages/crumbs/animate.py", line 5, in <module>
    display.start()
  File "/usr/local/lib/python3.8/dist-packages/pyvirtualdisplay/display.py", line 72, in start
    self._obj.start()
  File "/usr/local/lib/python3.8/dist-packages/pyvirtualdisplay/abstractdisplay.py", line 149, in start
    self._start1_has_displayfd()
  File "/usr/local/lib/python3.8/dist-packages/pyvirtualdisplay/abstractdisplay.py", line 197, in _start1_has_displayfd
    self.display = int(self._wait_for_pipe_text(rfd))
  File "/usr/local/lib/python3.8/dist-packages/pyvirtualdisplay/abstractdisplay.py", line 309, in _wait_for_pipe_text
    raise XStartTimeoutError(
pyvirtualdisplay.abstractdisplay.XStartTimeoutError: No reply from program Xvfb. command:['Xvfb', '-br', '-nolisten', 'tcp', '-screen', '0', '1280x1024x24', '-displayfd', '4']

Any idea on how to solve this?

ponty commented 2 years ago

Do you get the timeout after 10 minutes? Please read the previous messages and describe if your situation is the same or different than others. Do you have high load like this? https://github.com/ponty/PyVirtualDisplay/issues/62#issuecomment-884171201

Becheler commented 2 years ago

Yes, the timeout seems to happen after 10 minutes. For the load I am not sure about how to answer. The error is raised from a job I run from a singularity/docker image on an heterogeneous grid for distributed high throughput computing and I am not sure about how to pull information from there.

ponty commented 2 years ago

I can't reproduce the timeout. I added timeout parameter on master branch. retry parameter also works. You can find what works for your use case.