When I attempt to use a proxy server on the local configuration, and that proxy has authentication, Chromeless returns an empty body without an error.
I'm using the most recent Chromeless as of 14 July 2018. Here is my script.
I've obviously removed my private proxies usernames, passwords, and host names. If you message me 1:1 I'm happy to give them to you for testing purposes.
FYI, my proxies are provided by a commercial service where I pay for usage. I'm highly confident they are configured correctly. They have run millions of URLs without problems and continue to do so in our production environment.
const chromeLauncher = require('chrome-launcher');
const Chromeless = require('chromeless').Chromeless;
if (process.env.NODE_ENV !== 'production') {
require('dotenv').config()
}
const argv = require('minimist')(process.argv.slice(2));
const url = argv['u'];
async function run(url) {
const chrome = await chromeLauncher.launch({
port: 9222,
chromeFlags: [
'--disable-gpu',
'--headless',
//'--proxy-server=http://username:password@domain_name.com:80' // doesn't work (with DNS resolution to get IP)
//'--proxy-server=http://username:password@1.2.3.4:80' // doesn't work (checking DNS resolution as problem)
//'--proxy-server=http://201.49.66.97:92' // works (get one from https://www.proxynova.com/proxy-server-list/)
]
});
console.log('Chrome debuggable on port: ' + chrome.port);
const chromeless = new Chromeless({
launchChrome: false,
})
const html = await chromeless
.goto(url)
.wait(15000) // set arbitrarily long to ensure that this isn't a thing that its not finished rendering
.html()
const response = { html: html, url: url }
console.log(JSON.stringify(response))
await chromeless.end()
}
run(url).catch(console.error.bind(console))
Ideally, I can do this using Lambda as well. But, given its lack of "on demand changing proxy servers", I have opted to run a local Chrome for now.
Chrome Version: Version 67.0.3396.99 (Official Build) (64-bit)
Node Version: v10.5.0
Development Environment: MacOS High Sierra 10.13.5 (MBRt 16gb RAM)
When I attempt to use a proxy server on the local configuration, and that proxy has authentication, Chromeless returns an empty body without an error.
I'm using the most recent Chromeless as of 14 July 2018. Here is my script.
I've obviously removed my private proxies usernames, passwords, and host names. If you message me 1:1 I'm happy to give them to you for testing purposes.
FYI, my proxies are provided by a commercial service where I pay for usage. I'm highly confident they are configured correctly. They have run millions of URLs without problems and continue to do so in our production environment.
Ideally, I can do this using Lambda as well. But, given its lack of "on demand changing proxy servers", I have opted to run a local Chrome for now.
Chrome Version: Version 67.0.3396.99 (Official Build) (64-bit) Node Version: v10.5.0 Development Environment: MacOS High Sierra 10.13.5 (MBRt 16gb RAM)