apify / apify-actor-docker

Base Docker images for Apify actors.
https://hub.docker.com/u/apify
Apache License 2.0
70 stars 23 forks source link

XVFB required for Headless mode - apify/actor-node-playwright-chrome #139

Open JoshuwayMorris opened 2 months ago

JoshuwayMorris commented 2 months ago

Packages information

Application first generated through the Crawlee CLI.

This issue is related to the apify/actor-node-playwright-chrome Docker image.

Background

Dear team, I've recently been using Crawlee with Playwright and Headless Chrome. I deployed my project as a Dockerfile to DigitalOcean's App Platform. I followed the Running in Docker guide on the Crawlee website and copied the example "Browser and TypeScript" Dockerfile.

In this example Dockerfile, the following snippet is featured at the end:

# Run the image. If you know you won't need headful browsers,
# you can remove the XVFB start script for a micro perf gain.
CMD ./start_xvfb_and_run_cmd.sh && npm run start:prod --silent

I only want to run Crawlee using Playwright and headless Chrome, so I decided to follow the suggestion to remove the XVFB start script for a micro-perf gain.

However, when I removed this script, Chrome was unable to launch. Crawlee itself does not provide any Chrome-related error information. So, to try and work out what was going wrong, I copied the chromeTest script from this repository, and the error message included below was generated:

Confusingly, the issue only occurs when deployed to a VPS or DigitalOcean's App Platform. I'm not sure why this would be the case, as I thought the purpose of Docker was to avoid issues like this. If I run the same Docker image/container on my machine, I do not see the error.

For now, I have reinstated the ./start_xvfb_and_run_cmd.sh script into the final line of my Dockerfile, and the error no longer occurs.

Question

I am just wondering, is this a bug? Or if this is expected? As I want to run Chrome in Headless mode, should I need to start XVFB?

Thank you for providing this helpful collection of Docker images.

Error message when not starting XVFB

[example-crawler] [2024-08-29 16:32:27] node:internal/process/promises:391
[example-crawler] [2024-08-29 16:32:27]     triggerUncaughtException(err, true /* fromPromise */);
[example-crawler] [2024-08-29 16:32:27]     ^
[example-crawler] [2024-08-29 16:32:27] 
[example-crawler] [2024-08-29 16:32:27] Failed to launch browser. Please check the following:
[example-crawler] [2024-08-29 16:32:27] - Check whether the provided executable path "/usr/bin/google-chrome" is correct.
[example-crawler] [2024-08-29 16:32:27] - Try installing the required dependencies by running `npx playwright install --with-deps` (https://playwright.dev/docs/browsers).
[example-crawler] [2024-08-29 16:32:27] 
[example-crawler] [2024-08-29 16:32:27] The original error is available in the `cause` property. Below is the error received when trying to launch a browser:
[example-crawler] [2024-08-29 16:32:27] ​
[example-crawler] [2024-08-29 16:32:27] browserType.launchPersistentContext: Timeout 180000ms exceeded.
[example-crawler] [2024-08-29 16:32:27] Call log:
[example-crawler] [2024-08-29 16:32:27]   - <launching> /usr/bin/google-chrome --disable-field-trial-config --disable-background-networking --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --proxy-server=http://127.0.0.1:25326 --proxy-bypass-list=<-loopback> --disable-blink-features=AutomationControlled --user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 --user-data-dir=/tmp/playwright_chromiumdev_profile-jdQTAY --remote-debugging-pipe about:blank
[example-crawler] [2024-08-29 16:32:27]   - <launched> pid=25
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] Old Headless mode will be removed from the Chrome binary soon. Please use the new Headless mode (https://developer.chrome.com/docs/chromium/new-headless) or the chrome-headless-shell which is a standalone implementation of the old Headless mode (https://developer.chrome.com/blog/chrome-headless-shell).
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] 
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.146338:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.168675:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.203410:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.205878:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.244687:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.247093:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.248422:ERROR:address_tracker_linux.cc(235)] Could not bind NETLINK socket: Permission denied (13)
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.556816:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.558080:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.558673:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.559070:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.559706:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.560818:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.576332:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.576651:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.577166:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.577613:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.578189:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.578550:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162928.664190:ERROR:viz_main_impl.cc(166)] Exiting GPU process due to errors during initialization
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162929.353326:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.149396:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.151700:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.152161:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.152397:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.152721:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.153013:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.166530:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.166807:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.167053:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.167254:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.167932:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.168294:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.239108:ERROR:viz_main_impl.cc(166)] Exiting GPU process due to errors during initialization
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.431961:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.573406:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.573860:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.574999:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.575330:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.575572:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.576029:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.587134:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.587607:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.587804:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.588214:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.588616:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.588762:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]   - [pid=25][err] [0829/162930.652380:ERROR:viz_main_impl.cc(166)] Exiting GPU process due to errors during initialization
[example-crawler] [2024-08-29 16:32:27] 
[example-crawler] [2024-08-29 16:32:27]     at async PlaywrightPlugin._launch (/home/myuser/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:106:40)
[example-crawler] [2024-08-29 16:32:27]     at async testChrome (/home/myuser/dist/chromeTest.js:13:21)
[example-crawler] [2024-08-29 16:32:27] Error thrown at:
[example-crawler] [2024-08-29 16:32:27] 
[example-crawler] [2024-08-29 16:32:27]     at PlaywrightPlugin._throwAugmentedLaunchError (/home/myuser/node_modules/@crawlee/browser-pool/abstract-classes/browser-plugin.js:153:15)
[example-crawler] [2024-08-29 16:32:27]     at PlaywrightPlugin._throwOnFailedLaunch (/home/myuser/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:164:14)
[example-crawler] [2024-08-29 16:32:27]     at /home/myuser/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:109:33
[example-crawler] [2024-08-29 16:32:27]     at async PlaywrightPlugin._launch (/home/myuser/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:106:40)
[example-crawler] [2024-08-29 16:32:27]     at async testChrome (file:///home/myuser/dist/chromeTest.js:13:21) {
[example-crawler] [2024-08-29 16:32:27]   name: 'BrowserLaunchError',
[example-crawler] [2024-08-29 16:32:27]   [cause]: browserType.launchPersistentContext: Timeout 180000ms exceeded.
[example-crawler] [2024-08-29 16:32:27]   Call log:
[example-crawler] [2024-08-29 16:32:27]     - <launching> /usr/bin/google-chrome --disable-field-trial-config --disable-background-networking --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate,HttpsUpgrades,PaintHolding,PlzDedicatedWorker --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --disable-search-engine-choice-screen --unsafely-disable-devtools-self-xss-warnings --headless=old --hide-scrollbars --mute-audio --blink-settings=primaryHoverType=2,availableHoverTypes=2,primaryPointerType=4,availablePointerTypes=4 --no-sandbox --proxy-server=http://127.0.0.1:25326 --proxy-bypass-list=<-loopback> --disable-blink-features=AutomationControlled --user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 --user-data-dir=/tmp/playwright_chromiumdev_profile-jdQTAY --remote-debugging-pipe about:blank
[example-crawler] [2024-08-29 16:32:27]     - <launched> pid=25
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] Old Headless mode will be removed from the Chrome binary soon. Please use the new Headless mode (https://developer.chrome.com/docs/chromium/new-headless) or the chrome-headless-shell which is a standalone implementation of the old Headless mode (https://developer.chrome.com/blog/chrome-headless-shell).
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] 
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.146338:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.168675:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.203410:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.205878:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.244687:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.247093:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.248422:ERROR:address_tracker_linux.cc(235)] Could not bind NETLINK socket: Permission denied (13)
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.556816:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.558080:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.558673:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.559070:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.559706:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.560818:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.576332:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.576651:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.577166:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.577613:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.578189:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.578550:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162928.664190:ERROR:viz_main_impl.cc(166)] Exiting GPU process due to errors during initialization
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162929.353326:WARNING:bluez_dbus_manager.cc(248)] Floss manager not present, cannot set Floss enable/disable.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.149396:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.151700:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.152161:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.152397:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.152721:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.153013:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.166530:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.166807:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.167053:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.167254:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.167932:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.168294:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.239108:ERROR:viz_main_impl.cc(166)] Exiting GPU process due to errors during initialization
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.431961:ERROR:file_path_watcher_inotify.cc(890)] Failed to read /proc/sys/fs/inotify/max_user_watches
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.573406:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.573860:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.574999:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.575330:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.575572:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.576029:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.587134:ERROR:angle_platform_impl.cc(44)] DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.587607:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.587804:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.588214:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.588616:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.588762:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.
[example-crawler] [2024-08-29 16:32:27]     - [pid=25][err] [0829/162930.652380:ERROR:viz_main_impl.cc(166)] Exiting GPU process due to errors during initialization
[example-crawler] [2024-08-29 16:32:27]   
[example-crawler] [2024-08-29 16:32:27]       at async PlaywrightPlugin._launch (/home/myuser/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:106:40)
[example-crawler] [2024-08-29 16:32:27]       at async testChrome (/home/myuser/dist/chromeTest.js:13:21) {
[example-crawler] [2024-08-29 16:32:27]     name: 'TimeoutError'
[example-crawler] [2024-08-29 16:32:27]   }
[example-crawler] [2024-08-29 16:32:27] }
[example-crawler] [2024-08-29 16:32:27] 
[example-crawler] [2024-08-29 16:32:27] Node.js v20.17.0
B4nan commented 2 months ago

Run the image. If you know you won't need headful browsers, you can remove the XVFB start script for a micro perf gain.

I am not sure if this comment is still valid, it has been there for ages.

If you can't run it without X server, I can imagine there is no much we can do, and you should just leave the CMD as it was.

cc @vladfrangu

JoshuwayMorris commented 2 months ago

Thanks @B4nan, I thought a similar thing, that if it doesn't run without it, I must need it!

I thought I would post it as an issue in case anyone has a similar problem in the future.

B4nan commented 2 months ago

Yeah thanks for letting us know, we'll take a closer look and remove/adjust the comments if its really like that.

B4nan commented 2 months ago

Hmm, it seems to be working fine on apify platform even without X server, used a fresh playwright/chrome template (from apify CLI) and only removed that X server part from CMD. Trying the crawlee template now.

B4nan commented 2 months ago

Hmm the crawlee template indeed fails without the X server running, but the apify one works fine.