ChromeDriver doesn't work in headless mode on Linux

annaerdi commented 2 months ago

Hi!

When I run ghosts in "isheadless": "false" mode, everything works fine. But when I use "isheadless": "true", I get a session not created: DevToolsActivePort file doesn't exist error.

I'm using Ubuntu 22.04. I have installed the latest Chrome and its corresponding ChromeDriver. They are both in the ghosts folder.

This is my timeline.json file:

{
  "Id": "c3416428-99df-43b6-8370-046760da961f",
  "Status": "Run",
  "TimeLineHandlers": [
    {
      "HandlerType": "BrowserChrome",
      "Initial": "about:blank",
      "UtcTimeOn": "00:00:00",
      "UtcTimeOff": "00:00:00",
      "HandlerArgs": {
        "executable-location": "~/ghosts-experiment/ghosts-client-linux-v8.0.0/chrome-linux64",
        "isheadless": "true",
        "blockimages": "false",
        "blockstyles": "false",
        "blockflash": "false",
        "blockscripts": "true",
        "stickiness": 0,
        "stickiness-depth-min": 5,
        "stickiness-depth-max": 10000,
        "incognito": "false",
        "command-line-args": [
          "--ignore-certificate-errors"
        ]
      },
      "Loop": true,
      "TimeLineEvents": [
        {
          "Command": "random",
          "CommandArgs": [
            "http://www.ceoexpress.com",
            "http://wikipedia.org",
            "http://reddit.com",
            "http://instagram.com",
            "http://imdb.com",
            "http://huffingtonpost.com",
            "http://nytimes.com",
            "http://msn.com"
          ],
          "DelayAfter": 15000,
          "DelayBefore": 0
        }
      ],
      "ScheduleType": "Other"
    }
  ]
}

This is the app.log file:

2024/09/23 22:16:46.855|ghosts.client.linux.Program.Run|Initiating ghosts.client.linux startup - Local: 00:16:46.8326007 UTC: 22:16:46.8536619
2024/09/23 22:16:47.061|Ghosts.Domain.Code.ClientConfigurationLoader.get_Config|App config loaded successfully: /home/anna/ghosts-experiment/ghosts-client-linux-v8.0.0/config/application.json
2024/09/23 22:16:47.061|ghosts.client.linux.Program.Run|Sockets enabled. Connecting...
2024/09/23 22:16:47.061|ghosts.client.linux.Comms.CheckId..ctor|CheckId instantiated with ID: 172cb37f-43ef-4a4f-ae77-ad0d1b2dc68e
2024/09/23 22:16:47.061|ghosts.client.linux.Program.Run|CheckID: 172cb37f-43ef-4a4f-ae77-ad0d1b2dc68e
2024/09/23 22:16:47.075|ghosts.client.linux.Program.Run|Survey disabled, continuing.
2024/09/23 22:16:47.075|ghosts.client.linux.Program.Run|Handlers enabled, initalizing...
2024/09/23 22:16:47.143|ghosts.client.linux.timelineManager.Orchestrator.Run|Stopfile watcher is starting
2024/09/23 22:16:47.151|ghosts.client.linux.timelineManager.Orchestrator.ThreadLaunch|Attempting new thread for: BrowserChrome
2024/09/23 22:16:47.229|ghosts.client.linux.Comms.Updates.GetServerUpdates|09/24/2024 00:16:47 - No new configuration found
2024/09/23 22:16:47.691|ghosts.client.linux.handlers.BrowserChrome..ctor|System.InvalidOperationException: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /home/anna/.cache/selenium/chrome/linux64/129.0.6668.58/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (SessionNotCreated)

After some googling, I tried adding these option, but they don't solve the issue either:

"command-line-args": [
  "--no-sandbox",
  "--disable-dev-shm-usage"
]

sei-dupdyke commented 2 months ago

Very strange days on Ubuntu (with this and some other issues on newer versions).

Could there be missing dependencies that Chrome requires in a headless environment? I am not sure, but though the graphical version works, headless mode may require additional packages or settings to function properly.

Can you try running chrome headless directly please? Should be something like this:

/path/to/your/chrome --headless https://www.google.com

annaerdi commented 2 months ago

It seems to be a chrome issue indeed. Running it directly resulted in the following error:

$ ./chrome-linux64/chrome --headless https://www.google.com
chrome_crashpad_handler: --database is required
Try 'chrome_crashpad_handler --help' for more information.
[5651:5651:0925/103029.921387:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)
Trace/breakpoint trap (core dumped)

After going through these steps trying to debug it unsuccessfully, I have installed a previous version of chrome and chromedriver, which works fine.

(I'll leave them here in case someone comes accross the same issue: Chrome: https://storage.googleapis.com/chrome-for-testing-public/125.0.6422.60/linux64/chrome-linux64.zip Chromedriver: https://storage.googleapis.com/chrome-for-testing-public/125.0.6422.60/linux64/chromedriver-linux64.zip)

However, I encountered some strange behavior while experimenting with different Chrome versions:

First, I installed an older version and deleted the newer one, which I don't want to use.
I passed the location of the older version to the executable-location parameter, but it was still using the newer version. Fair enough, since the /home/anna/.cache/selenium/chromedriver/linux64/129.0.6668.59/chromedriver folder still existed, so it probably just took it from the cache.
But then, I removed that folder, and yet it recreated it with the newer version, which doesn't exist anymore.
Even more strangely, I completely deleted every Chrome, Chromedriver, and all the caches (both /home/anna/.cache/selenium/ and /root/.cache/selenium/ folders), yet GHOSTS was still able to start Chrome, and it recreated the /home/anna/.cache/selenium/ folder.

Is it possible that GHOSTS is still caching an instance of chrome somewhere else?

sei-dupdyke commented 2 months ago

Thanks for all the detail!

Ghosts shouldn't cache (if it does, I am unaware how that is happening) anything like this.

There is a setting in ./config/application.json for "FirefoxInstallLocation" - you can try setting this with an absolute path, otherwise Ghosts looks in

/bin/firefox /usr/bin/firefox

for the FF binaries. It shouldn't be accessing chrome from anywhere else.

annaerdi commented 2 months ago

I think I found what is causing this behaviour: https://www.selenium.dev/blog/2023/whats-new-in-selenium-manager-with-selenium-4.11.0/

As of Selenium 4.11, it automatically downloads Chrome if it is not already installed on your system. This feature manages both drivers (like chromedriver) and browser versions. It seems that the executable-location parameter is being overridden by Selenium.

annaerdi commented 2 months ago

And another update: the original issue, which I had with Chrome version 129.0.6668.58, is not present anymore with the latest version 129.0.6668.70.

So, long story short: we actually don't have to bother anymore with manually installing Chrome :)

sei-dupdyke commented 2 months ago

Sorry, I was crossing firefox and chrome wires!!

cmu-sei / GHOSTS

ChromeDriver doesn't work in headless mode on Linux #414