search

TheBrainFamily / chimpy

Develop acceptance tests & end-to-end tests with realtime feedback.
https://thebrainfamily.github.io/chimpy
MIT License
52 stars 28 forks source link

How to get Chrome headless to download files? #108

Open samhatoum opened 6 years ago

samhatoum commented 6 years ago

Issue by rsshilli Wednesday Feb 07, 2018 at 06:16 GMT Originally opened as https://github.com/xolvio/chimp/issues/679

How do I download files using Chrome headless? I see that:

I'm having a hard time piecing together how I get this working from my Chimp world.

Versions:

I've tried a number of things playing with the configuration, trying things like this below, but nothing works:

chimpOptions.webdriverio.desiredCapabilities = {
  chromeOptions: {
    args : ["--no-sandbox", "--headless", "--disable-gpu", "--window-size=1050,900"],
    prefs: {
      "credentials_enable_service": false,
      "Browser.setDownloadBehavior": {behavior : "allow", downloadPath: path.join((process.env.HOME || process.env.HOMEPATH || process.env.USERPROFILE), "Downloads")},
      "profile": {
        "password_manager_enabled": false,
        "default_content_setting_values": { "automatic_downloads": 1 }
      }
    }
  },
};

My mechanism for downloading a file is to click on the download link and then use fs to wait for the file on the file system, which never shows up :(. This worked fine before adding --headless

samhatoum commented 6 years ago

Comment by ybatsiun Monday May 14, 2018 at 10:11 GMT

I am facing the same issue, were you been able to solve it?

samhatoum commented 6 years ago

Comment by ivoneijr Tuesday May 15, 2018 at 04:06 GMT

Same issue.

samhatoum commented 6 years ago

Comment by pelly Thursday May 24, 2018 at 16:08 GMT

Any progress on this? I'm having the same issue.

samhatoum commented 6 years ago

Comment by rsshilli Thursday May 24, 2018 at 19:21 GMT

I have this working, but I did it without the "--headless" mode. I was previously under the impression that the only way to make Chimp test Chrome in a docker container was the use the headless mode. I was wrong. Just remove the headless and everything works fine, even when in a docker container or running on a machine without a monitor.

samhatoum commented 6 years ago

Comment by HolyWalley Wednesday May 30, 2018 at 09:30 GMT

@rsshilli did you ever get an error message like this

unknown error: DevToolsActivePort file doesn't exist (Driver info: chromedriver=2.39.562737 (dba483cee6a5f15e2e2d73df16968ab10b38a2bf),platform=Linux 4.4.0-1049-aws x86_64)

when you run not headless chrome in docker-container?

samhatoum commented 6 years ago

Comment by pedymaster Monday Jun 04, 2018 at 08:23 GMT

@rsshilli Iam interested in this too. I have the same error as HolyWalley when removing the headless flag

samhatoum commented 6 years ago

Comment by rsshilli Monday Jun 04, 2018 at 20:33 GMT

You have to pass in the full path to the file, right? It looks like from the error message that you just passed in the filename itself (DevToolsActivePort). The browser won't know where to find it.

samhatoum commented 6 years ago

Comment by wuthiago Wednesday Jul 18, 2018 at 18:23 GMT

I am also facing this problem. Anyone able to make it work? Thanks

samhatoum commented 6 years ago

Comment by rsshilli Thursday Jul 19, 2018 at 13:12 GMT

@wuthiago I made it work. Are you using the full path for the filename?

samhatoum commented 6 years ago

Comment by wuthiago Thursday Jul 19, 2018 at 13:34 GMT

@rsshilli were you able to download files with the "--headless" mode on?

samhatoum commented 6 years ago

Comment by rsshilli Thursday Jul 19, 2018 at 13:50 GMT

No, I never tried that, TBH.

On Thu, Jul 19, 2018 at 9:34 AM, wuthiago notifications@github.com wrote:

@rsshilli https://github.com/rsshilli were you able to download files with the "--headless" mode on?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xolvio/chimp/issues/679#issuecomment-406277946, or mute the thread https://github.com/notifications/unsubscribe-auth/ACXzBRAODZymhQ1-bXVd62HfDqkpKaePks5uIIrngaJpZM4R8Mnm .

-- Ryan Shillington CTO

CherryCircle Software, Inc. 1701 Trinity Street https://maps.google.com/?q=1701+Trinity+Street+Austin,+TX+78712&entry=gmail&source=g Austin, TX 78712 https://maps.google.com/?q=1701+Trinity+Street+Austin,+TX+78712&entry=gmail&source=g Direct: 512.426.4389 www.cherrycirclesoftware.com www.qbdvision.com

samhatoum commented 6 years ago

Comment by komalanandpandey Thursday Jul 19, 2018 at 14:52 GMT

Hi, did any one get luck on downloading file using "headless chrome" , please help! Regards!

samhatoum commented 6 years ago

Comment by pelly Thursday Jul 19, 2018 at 14:59 GMT

No, I gave up and used xvfb. Headless chrome had all kinds of problems for me.

On Thu, Jul 19, 2018 at 10:52 AM, komalanandpandey <notifications@github.com

wrote:

Hi, did any one get luck on downloading file using "headless chrome" , please help! Regards!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/xolvio/chimp/issues/679#issuecomment-406304379, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKQmFOp-quMdn3JVk-I-0ZAsU-NMPdNks5uIJ0bgaJpZM4R8Mnm .

samhatoum commented 6 years ago

Comment by c20xh2 Friday Jul 20, 2018 at 08:39 GMT

@komalanandpandey @wuthiago

Here's a solution that work for me :

Setting the driver options:

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
prefs = {'download.default_directory' : out_path}
options.add_experimental_option('prefs', prefs)

download_path = './output/'

driver = webdriver.Chrome('./chromedriver', chrome_options=options)

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_path}}
command_result = driver.execute("send_command", params)

Downloading a file:

driver.get(url)
time.sleep(20)

IMPORTANT : You have to call time.sleep(x) to let some time for the download to finish, if the driver close before the end of file download, solution won't work

aasimali commented 5 years ago

@samhatoum

Can you tell me, please? how exactly to update the driver options? to enable the download option in the headless browser?

I use Ubuntu 16.04 LTS Thank you.

Elangopalakrishnan commented 5 years ago

@samhatoum

How we can implement above solution in c# script coz i need to download pdf and validate in chrome headless browser mode..

Please guide me on this..

Thanks in advance, Elango

hugochibougamau commented 5 years ago

To add more noise to the conversation. Using groovyVersion = '2.5.4' gebVersion = '2.3' seleniumVersion = '3.141.59' chromeDriverVersion = '2.45' geckoDriverVersion = '0.24.0' ieDriverVersion = '3.6'

the chromeHeadless driver is unable to download a file. However, everything is fine using Chrome (Version 71.0.3578.98) or by that means Firefox.

The parameters that I am using to invoke chromeHeadless are (in GebConfig) are

chromeHeadless {
    driver = {
        ChromeOptions o = new ChromeOptions()
        o.addArguments('headless')
        o.addArguments('disable-gpu') 
        o.addArguments('no-sandbox')
        o.addArguments('window-size=1980,1080')
        new ChromeDriver(o)
    }
}

May this be caused by a bug in the chromeHeadless driver implementation?

As the previous users have indicated, any help will be very welcomed

OliverScholle commented 5 years ago

For our C# environment we use the packages: Selenium.Chrome.WebDriver v2.35.0 Selenium.WebDriver v3.10.0 Appium.WebDriver v4.0.0.1-beta Chrome Version 72.0.3626.121

{
     var options = new ChromeOptions();
     options.AddArguments("headless", "disable-gpu");
     var driver = new ChromeDriver(chromedriverPath, options);
     var param = new Dictionary<string, object>();
     param.Add("behavior", "allow");
     param.Add("downloadPath", $"C:\\Users\\{Environment.UserName}\\Downloads\\");
     driver.ExecuteChromeCommand("Page.setDownloadBehavior", param);
}

We also had the same problem of Excel files not being downloaded after being clicked. Once the behavior parameter was allowed for the Page.setDownloadBehavior command it just worked immediately.

vanquy72dn commented 5 years ago

Is any available workaround to overcome this issue with Java/Javascript? Thanks so much.

lucasczpnk commented 5 years ago

For those who are struggling with this feature in python:

I've found a great headless chrome driver builder made by @shawnbutton and it's available in his repo [PythonHeadlessChrome].

[found at stackoverflow]

UsharaniA commented 5 years ago

Can any one help me with JavaScript and am surfing the internet for last 2 days but not getting any solution.

Am using WebdriverIO tool and below is my configuration

chromeOptions: {

        args: ['--headless', '--disable-gpu', '--window-size=1366,768'],
        prefs: {    
           // 'download.default_directory': $'C:\\Users\\n276546\\Downloads',
            'download.prompt_for_download':"false",
            'download.directory_upgrade':"true",
            'Page.set_download_behavior': { 'behavior': 'allow', 'downloadPath': '${workspaceRoot}\\resource' },
          },

    }
TC-rGangal commented 5 years ago

facing the same issue with webdriverio : here's what I am doing : 

var path = require('path')
var DownloadFldr = path.resolve('./test/download/');
.
.
.
browser.sendCommand('Page.setDownloadBehavior', {'behavior': 'allow', 'downloadPath': DownloadFldr})

chrome options: 

{ browserName: 'chrome',
  'goog:chromeOptions':
   { args:
      [ '--headless',
        '--disable-gpu',
        '--window-size=1920,1080',
        '--no-sandbox',
        '--disable-dev-shm-usage',
        '--disable-software-rasterizer',
        '--disable-popup-blocking',
        '--disable-extensions',
        ' --lang\\=en-us',
        '--test-type' ],
     binary:
      '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
     prefs:
      { 'download.default_directory': '/Users/**/**/test/download' } } }
UsharaniA commented 5 years ago

I get browser.sendCommand is not a function.

UsharaniA commented 5 years ago

Finally i could able to download using webdriverIO.

const unirest = require('unirest');

let session = browser.session(); // key sessionId is webdriverio implementation, debug your session object let sessionId = session['sessionId']; let params = { 'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': 'C:\gitNodejs\art-automation-test\CucumberWebdriverIO\testResults\temp'}};

unirest .post('http://localhost:4444/wd/hub/session/' + sessionId + '/chromium/send_command') .send(JSON.stringify(params)) .end();

My Config file:

browserName: 'chrome', chromedriverVersion: '2.46', chromeOptions: {

        args: ['--headless', '--disable-gpu', '--window-size=1366,768','--disable-web-security','--no-sandbox','--incognito'],
        prefs: {    
            'download.default_directory': 'C:\\gitNodejs\\art-automation-test\\CucumberWebdriverIO\\testResults\\temp',
            'download.prompt_for_download':false,
            'download.directory_upgrade':true,
            "safebrowsing.enabled":false,
            "safebrowsing.disable_download_protection":true
          },

    }
asilvis commented 5 years ago

none of these make it work, but this is the combination for WebdriverIO (JS) that worked out.

capabilities: [{
        maxInstances: 5,
        browserName: 'chrome',

        'goog:chromeOptions': {
            args: ['--disable-gpu', '--disable-web-security','--no-sandbox'],
            prefs: {    
                'download.default_directory': downloadDir,
                'download.prompt_for_download':false,
                'download.directory_upgrade':true,
                "safebrowsing.enabled":false,
                "safebrowsing.disable_download_protection":true
            },
        }
    }],

and:

   before: function (capabilities, specs) {
        browser.sendCommand('Page.setDownloadBehavior', {'behavior': 'allow', 'downloadPath': downloadDir})
        let session = browser.getSession();
     },

Hope this help a lonely soul :).

megagon commented 5 years ago

for anyone who is experiencing this for wdiov5: chrome capabilities (wdio.conf)

capabilities: [{
    maxInstances: 5,
    browserName: 'chrome',
    'goog:chromeOptions': {
      args: ['--headless', '--disable-gpu',
        '--window-size=1280,800',
        '--proxy-server=\'direct://\'',
        '--proxy-bypass-list=*',
        '--no-sandbox',
        '--disable-dev-shm-usage',
        '--user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"',
        '--remote-debugging-port=3434',
      ],
      prefs: {
        download: {
          default_directory: pathToDownload,
        },
      },
    },
  }],

devtools (wdio.conf) services: 

['selenium-standalone', ['devtools', {
    debuggerAddress: 'localhost:3434',
  }]],

and after that before I start download I run 

browser.cdp('Page', 'setDownloadBehavior', {
      behavior: 'allow',
      downloadPath: pathToDownload,
    });
pcamen commented 5 years ago

Ditto on using Xvfb with non-headless Chrome / Chromium-browser. We banged our heads against the wall for weeks trying to get --headless to work with downloads. Here is specifically what we did:

  1. Install Xvfb sudo apt-get install xvfb
  2. Start a screen session
  3. Start Xvfb process to provide a virtual display Xvfb :1 -screen 5 1024x768x8 &
  4. Add DISPLAY to the environment in the screen session export DISPLAY=:1.5
  5. Start chromedriver (which will have the DISPLAY in its environment) chromedriver &

Now any chromium-browser processes started by chromedriver will inherit DISPLAY and will use the Xvfb virtual display.

damuz91 commented 5 years ago

Ditto on using Xvfb with non-headless Chrome / Chromium-browser. We banged our heads against the wall for weeks trying to get --headless to work with downloads. Here is specifically what we did:

  1. Install Xvfb sudo apt-get install xvfb
  2. Start a screen session
  3. Start Xvfb process to provide a virtual display Xvfb :1 -screen 5 1024x768x8 &
  4. Add DISPLAY to the environment in the screen session export DISPLAY=:1.5
  5. Start chromedriver (which will have the DISPLAY in its environment) chromedriver &

Now any chromium-browser processes started by chromedriver will inherit DISPLAY and will use the Xvfb virtual display.

@pcamen If we got a VPS with an automatic crawler i think this wont be a solution would it?

pcamen commented 5 years ago

We are using this on a Linode VPS but I don't see why it wouldn't work on any headless VPS. As for an automatic crawler, I assume you are talking about something you built yourself that uses Chrome, yes? We are using this with a system we built to do some integrations with an enterprise MRP software package that has a web interface that won't work with simpler scraping solutions, and has some of its data presented as downloadable report files. Works great for us this way.

xkasberg commented 5 years ago

For those who are struggling with this feature in python:

I've found a great headless chrome driver builder made by @shawnbutton and it's available in his repo [PythonHeadlessChrome].

[found at stackoverflow]

Did you actually try this? I tried this and did not get it to work.

jdmwood commented 5 years ago

I managed to get this working with protractor tests. Followed similar approach to above:

browser.driver.sendChromiumCommand('Page.setDownloadBehavior', {'behavior': 'allow', 'downloadPath': '/tmp'});

(browser.driver is just the native WebDriver driver object so this should work even if you're not using protractor).

The only difference for me was that driver.execute() didn't exist and neither did various other methods, but driver.sendChromiumCommand() did exist. Guess it depends on your version of webdriver...

PS: If you do the above then you don't seem to need any special download prefs in the Webdrive capabilities section (though no harm in adding them I guess to support users not running in headless).

lucasczpnk commented 5 years ago

@KristianTua Did you actually try this? I tried this and did not get it to work.

Hi Kristian, Sorry for the long delay!

And technically yes, I've added a few lines to it to better suit my needs. Here's my version, so you can test it:

#!/usr/bin/python3.6
# -*- coding: utf-8 -*-
# env
from pathlib import Path as plib
import sys
# web modules
from selenium.webdriver import Chrome
from selenium.webdriver.chrome import webdriver as chrome_webdriver

class WdriverBuilder(object):
    def get_wdriver(self, dir_path, download_location=None, headless=False):
        #
        wdriver = self._get_chrome_wdriver(dir_path, download_location, headless)
        # wdriver.set_window_size(1400, 700)
        #
        return wdriver

    def _get_chrome_wdriver(self, dir_path, download_location, headless):
        chrome_options = chrome_webdriver.Options()
        if download_location:
            prefs = {'download.default_directory': download_location,
                     'download.prompt_for_download': False,
                     'download.directory_upgrade': True,
                     'safebrowsing.enabled': False,
                     'safebrowsing.disable_download_protection': True}

            chrome_options.add_experimental_option('prefs', prefs)

        if headless:
            chrome_options.add_argument("--headless")

        dir_path = plib(dir_path)
        wdriver_path = str(dir_path / 'chromedriver')

        if sys.platform.startswith("win"):
            wdriver_path += ".exe"

        wdriver = Chrome(executable_path=wdriver_path, chrome_options=chrome_options)

        if headless:
            self.enable_download_in_headless_chrome(wdriver, download_location)

        return wdriver

    def enable_download_in_headless_chrome(self, wdriver, download_dir):
        """
        there is currently a "feature" in chrome where
        headless does not allow file download: https://bugs.chromium.org/p/chromium/issues/detail?id=696481
        This method is a hacky work-around until the official chromewdriver support for this.
        Requires chrome version 62.0.3196.0 or above.
        """
        # add missing support for chrome "send_command"  to selenium webwdriver
        wdriver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
        #
        params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
        command_result = wdriver.execute("send_command", params)
        print("\nresponse from browser:")
        for key in command_result:
            print("result:" + key + ":" + str(command_result[key]))
xkasberg commented 5 years ago

Hey Lucas,

Thanks so much for getting back to me ill give this a try.

Thanks!

[image: TUA_Logo (1).png]

Kristian Berg

Credit Analyst, tua Financial Technologies Ltd https://www.tua.ca/. http://www.tua.ca/

587-885-0447 <(587)%20885-0447>

[image: Facebook.png] http://www.facebook.com/tuaFinancial[image: Instagram.png] http://www.instagram.com/tuaFinancial

On Thu, Jul 18, 2019 at 8:21 AM Lucas Vasconcellos Czepaniki < notifications@github.com> wrote:

@KristianTua https://github.com/KristianTua Did you actually try this? I tried this and did not get it to work.

Hi Kristian, Sorry for the long delay!

And technically yes, I've added a few lines to it to better suit my needs. Here's my version, so you can test it:

!/usr/bin/python3.6# -- coding: utf-8 --# envfrom pathlib import Path as plibimport sys# web modulesfrom selenium.webdriver import Chromefrom selenium.webdriver.chrome import webdriver as chrome_webdriver

class WdriverBuilder(object): def get_wdriver(self, dir_path, download_location=None, headless=False): # wdriver = self._get_chrome_wdriver(dir_path, download_location, headless)

wdriver.set_window_size(1400, 700)

    #
    return wdriver

def _get_chrome_wdriver(self, dir_path, download_location, headless):
    chrome_options = chrome_webdriver.Options()
    if download_location:
        prefs = {'download.default_directory': download_location,
                 'download.prompt_for_download': False,
                 'download.directory_upgrade': True,
                 'safebrowsing.enabled': False,
                 'safebrowsing.disable_download_protection': True}

        chrome_options.add_experimental_option('prefs', prefs)

    if headless:
        chrome_options.add_argument("--headless")

    dir_path = plib(dir_path)
    wdriver_path = str(dir_path / 'chromedriver')

    if sys.platform.startswith("win"):
        wdriver_path += ".exe"

    wdriver = Chrome(executable_path=wdriver_path, chrome_options=chrome_options)

    if headless:
        self.enable_download_in_headless_chrome(wdriver, download_location)

    return wdriver

def enable_download_in_headless_chrome(self, wdriver, download_dir):
    """        there is currently a "feature" in chrome where        headless does not allow file download: https://bugs.chromium.org/p/chromium/issues/detail?id=696481        This method is a hacky work-around until the official chromewdriver support for this.        Requires chrome version 62.0.3196.0 or above.        """
    # add missing support for chrome "send_command"  to selenium webwdriver
    wdriver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
    #
    params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
    command_result = wdriver.execute("send_command", params)
    print("\nresponse from browser:")
    for key in command_result:
        print("result:" + key + ":" + str(command_result[key]))

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TheBrainFamily/chimpy/issues/108?email_source=notifications&email_token=ALT2I2F6NZNJ4YR7SHUHHEDQAB36VA5CNFSM4FLGQCN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2IUKPA#issuecomment-512836924, or mute the thread https://github.com/notifications/unsubscribe-auth/ALT2I2BXDZMRNOI7F27UWFLQAB36VANCNFSM4FLGQCNQ .

lucasczpnk commented 5 years ago

@KristianTua No problem!

Let me know if you need any help getting it to work.

vigyanhoon commented 4 years ago

`System.setProperty("webdriver.chrome.driver", path); DesiredCapabilities capabilities = DesiredCapabilities.chrome(); capabilities.setCapability(CapabilityType.UNEXPECTED_ALERT_BEHAVIOUR, UnexpectedAlertBehaviour.ACCEPT);

// For Custom Download Folder HashMap<String, Object> chromePrefs = new HashMap<>(); chromePrefs.put("profile.default_content_settings.popups", 0); chromePrefs.put("download.default_directory", downloadFilepath); chromePrefs.put("download.prompt_for_download", false); chromePrefs.put("download.directory_upgrade", true); chromePrefs.put("safebrowsing.enabled", false); chromePrefs.put("safebrowsing.disable_download_protection", true); chromePrefs.put("downloadPath", downloadFilepath); chromePrefs.put("behavior", "allow"); chromePrefs.put("profile.default_content_setting_values.notifications", 1); ChromeOptions options = new ChromeOptions(); options.setExperimentalOption("prefs", chromePrefs); options.setExperimentalOption("useAutomationExtension", true); if (FOSProperties.isHeadless()) { options.addArguments("--headless"); options.addArguments("--window-size=1920,1080"); options.addArguments("--remote-debugging-port=9999"); } options.addArguments("start-maximized"); options.addArguments("disable-infobars"); options.addArguments("--disable-extensions"); options.addArguments("--disable-gpu"); options.addArguments("--no-sandbox"); options.addArguments("--disable-dev-shm-usage"); capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true); capabilities.setCapability(CapabilityType.UNEXPECTED_ALERT_BEHAVIOUR, UnexpectedAlertBehaviour.IGNORE); capabilities.setCapability(ChromeOptions.CAPABILITY, options);

ChromeDriverService driverService = ChromeDriverService.createDefaultService(); driver = new ChromeDriver(driverService, capabilities);

Map<String, Object> commandParams = new HashMap<>(); commandParams.put("cmd", "Page.setDownloadBehavior"); commandParams.put("params", chromePrefs);

ObjectMapper objectMapper = new ObjectMapper(); HttpClient httpClient = HttpClientBuilder.create().build();

try { String command = objectMapper.writeValueAsString(commandParams); String url = driverService.getUrl().toString() + "/session/" + ((RemoteWebDriver) driver).getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(url);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

} catch (Exception e) { e.printStackTrace(); }`

Do you see something wrong with this JAVA code? Headless file download doesn't work for me.

xkasberg commented 4 years ago

@KristianTua No problem!

Let me know if you need any help getting it to work.

I could not get your code to work on Linux

xkasberg commented 4 years ago

@KristianTua No problem!

Let me know if you need any help getting it to work.

I could not get this to work on mac, chrome version 75 and chromedriver version 75

lucasczpnk commented 4 years ago

@KristianTua No problem! Let me know if you need any help getting it to work.

I could not get your code to work on Linux

Can you share the stack trace or any log you might have of the issue?

If memory serves me right, I also had an issue with v75 and since then I'm using 74. Try using v74 and if it still doesn't work, please share the stack trace or error message here, so that we can try to figure out the cause.

Here's the official link for this version's download: https://chromedriver.storage.googleapis.com/index.html?path=74.0.3729.6/

lucasczpnk commented 4 years ago

@vigyanhoon

Do you see something wrong with this JAVA code? Headless file download doesn't work for me.

I'm not really versed in Java, but I tried testing your code anyways. After trying to execute your code I got "No "public class" found to execute". As far as I know (I might be wrong, though), you need to have a main method inside a public class, except for servlets / applets i think. Could you share more information about your issue?

vigyanhoon commented 4 years ago

Its just chromedriver part of code I pasted. Will try to create an independent example.

vibhuist commented 4 years ago

Comment by c20xh2 Friday Jul 20, 2018 at 08:39 GMT

@komalanandpandey @wuthiago

Here's a solution that work for me :

Setting the driver options:

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
prefs = {'download.default_directory' : out_path}
options.add_experimental_option('prefs', prefs)

download_path = './output/'

driver = webdriver.Chrome('./chromedriver', chrome_options=options)

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_path}}
command_result = driver.execute("send_command", params)

Downloading a file:

driver.get(url)
time.sleep(20)

IMPORTANT : You have to call time.sleep(x) to let some time for the download to finish, if the driver close before the end of file download, solution won't work

Works!

shwetaneelsharma commented 4 years ago

How do I get this working in Behat with Mink?

mvgiacomello commented 4 years ago

Literally, just like @jdmwood2 said. I made it work on Webdriver.IO V5 with chrome headless by simply adding the following code to the before hook:

before: function(capabilities, specs) {
    browser.sendCommand('Page.setDownloadBehavior', {
      behavior: 'allow',
      downloadPath: <my download directory>,
    });
  },

I did not add absolutely no prefs.

vidya-sundararaman-q2 commented 4 years ago

@mvgiacomello Can you tell me which chrome and selenium versions you are using. I have not been able to get it working with "selenium-standalone": "^6.17.0" This is what i have in my before function browser.sendCommand('Page.setDownloadBehavior', { behavior: 'allow', downloadPath: downloadDir }) This is my chrome settings browserName: 'chrome', 'goog:chromeOptions': { prefs: { 'download.directory_upgrade': true, 'download.prompt_for_download': false, 'download.default_directory': downloadDir, 'safebrowsing.enabled': false, 'safebrowsing.disable_download_protection': true },

    args: ['--headless', '--disable-gpu', '--window-size=2048,1280']
  }
mmacvicar-splunk commented 4 years ago

TL;DR upgrading to chromedriver 79.0.0 on OSX allowed this to work for me.

I was banging my head against this on OSX using the javascript selenium-webdriver and chromedriver 78.0.0 and couldn't get it to work. I found this bug about headless chromedriver not working on OSX https://bugs.chromium.org/p/chromium/issues/detail?id=979847, which was marked recently as "no longer reproducible". So I upgraded to chromedriver 79.0.0 and it started working for me.

My chromeOptions:

        "chromeOptions": {
          "args": [
            "--disable-gpu",
            "--no-sandbox",
            "--disable-setuid-sandbox",
            "--disable-dev-shm-usage",
            "--disable-extensions",
            "--headless"
          ],
          "prefs": {
            "download": {
              "prompt_for_download": false,
              "directory_upgrade": true,
              "default_directory": "/tmp/downloads"
            },
            "browser": {
              "setDownloadBehavior": {
                "behavior": "allow",
                "downloadPath": "/tmp/downloads"
              }
            }
          }
bajins commented 4 years ago

Comment by c20xh2 Friday Jul 20, 2018 at 08:39 GMT

@komalanandpandey @wuthiago

Here's a solution that work for me :

Setting the driver options:

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
prefs = {'download.default_directory' : out_path}
options.add_experimental_option('prefs', prefs)

download_path = './output/'

driver = webdriver.Chrome('./chromedriver', chrome_options=options)

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_path}}
command_result = driver.execute("send_command", params)

Downloading a file:

driver.get(url)
time.sleep(20)

IMPORTANT : You have to call time.sleep(x) to let some time for the download to finish, if the driver close before the end of file download, solution won't work

driver.execute_cdp_cmd("Page.setDownloadBehavior", {'behavior': 'deny', 'downloadPath': ""})

https://github.com/SeleniumHQ/selenium/issues/5722#issuecomment-459161468

raghavmayank commented 4 years ago

Comment by c20xh2 Friday Jul 20, 2018 at 08:39 GMT

@komalanandpandey @wuthiago

Here's a solution that work for me :

Setting the driver options:

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
prefs = {'download.default_directory' : out_path}
options.add_experimental_option('prefs', prefs)

download_path = './output/'

driver = webdriver.Chrome('./chromedriver', chrome_options=options)

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_path}}
command_result = driver.execute("send_command", params)

Downloading a file:

driver.get(url)
time.sleep(20)

IMPORTANT : You have to call time.sleep(x) to let some time for the download to finish, if the driver close before the end of file download, solution won't work

after wasting so much time this worked. thanks :+1:

vipulgupta2048 commented 3 years ago

For anyone using Chrome Devtools protocol with WebdriverIO, this is the solution that actually works with headless Chrome/Chromium with WebdriverIo v6.

lhorvath87 commented 3 years ago

Hello Guys,

I had same problem (Windows 10) in python. If you have in your code this solution:

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_path}}
command_result = driver.execute("send_command", params)

You can try to change your download_path variable I had it like this:

download_path = "C:/Users/my user name/Downloads"

Did not work and I changed it to:

download_path = "C:\\Users\\my user name\\Downloads"

Changing backslashes worked for me. Hope it will help you too.

nerrante commented 3 years ago

@lhorvath87 THANK YOU for that - I am not even using Chimpy but was troubleshooting why headless chrome wasn't downloading for me using another tool and found your answer- it was all about the download path and the slashes. I changed that and it began working.