ulixee / secret-agent

The web scraper that's nearly impossible to block - now called @ulixee/hero
https://secretagent.dev
MIT License
670 stars 44 forks source link

create browser not working #38

Closed neryash closed 4 years ago

neryash commented 4 years ago

SecretAgent.createBrowser is not a function

blakebyrnes commented 4 years ago

Could you share more about your script and how you're encountering this error?

neryash commented 4 years ago

const secretAgent = require('secret-agent');

(async () => { const browser = await secretAgent.createBrowser(); await browser.goto('https://example.org'); const title = await browser.document.querySelector('title').textContent; const intro = await browser.document.querySelector('p:first-child').textContent; console.log(title); console.log(intro); await browser.close();
})();

I ran the script from your docs, but it crashed... I just ran it with using node app.js

blakebyrnes commented 4 years ago

Ok, I see what's happening. We need to update that example. Please try the script this way:

const secretAgent = require('secret-agent').default;

(async () => {
const browser = await secretAgent.createBrowser();
await browser.goto('https://example.org');
const title = await browser.document.querySelector('title').textContent;
const intro = await browser.document.querySelector('p').textContent;
console.log(title);
console.log(intro);
await browser.close();
})();
neryash commented 4 years ago

OK, now I am getting a new error: 2020-07-22T16:41:47.399Z ERROR SocketConnectDriver.ChildConnectError { error: Error: spawn C:\Users\nerya\Desktop\Fiverr\Gigs\Programming\Customers\keadam\node_modules\@secret-agent\mitm\lib/../socket/connect.exe ENOENT at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19) at onErrorNT (internal/child_process.js:469:16) at processTicksAndRejections (internal/process/task_queues.js:84:21) { errno: 'ENOENT', code: 'ENOENT', syscall: 'spawn C:\Users\nerya\Desktop\Fiverr\Gigs\Programming\Customers\keadam\node_modules\@secret-agent\mitm\lib/../socket/connect.exe', path: 'C:\Users\nerya\Desktop\Fiverr\Gigs\Programming\Customers\keadam\node_modules\@secret-agent\mitm\lib/../socket/connect.exe', spawnargs: [ '/tmp/sa-mitm-1.sock', '{"host":"example.org","port":"443","isSsl":true,"servername":"example.org","clientHelloId":"Chrome72","tcpTtl":64,"tcpWindowSize":65535}' ] } } 2020-07-22T16:41:47.404Z ERROR MitmHttpError { errorKind: 'PROXY_TO_SERVER_REQUEST_ERROR', error: Error: spawn C:\Users\nerya\Desktop\Fiverr\Gigs\Programming\Customers\keadam\node_modules\@secret-agent\mitm\lib/../socket/connect.exe ENOENT at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19) at onErrorNT (internal/child_process.js:469:16) at processTicksAndRejections (internal/process/task_queues.js:84:21) { errno: 'ENOENT', code: 'ENOENT', syscall: 'spawn C:\Users\nerya\Desktop\Fiverr\Gigs\Programming\Customers\keadam\node_modules\@secret-agent\mitm\lib/../socket/connect.exe', path: 'C:\Users\nerya\Desktop\Fiverr\Gigs\Programming\Customers\keadam\node_modules\@secret-agent\mitm\lib/../socket/connect.exe', spawnargs: [ '/tmp/sa-mitm-1.sock', '{"host":"example.org","port":"443","isSsl":true,"servername":"example.org","clientHelloId":"Chrome72","tcpTtl":64,"tcpWindowSize":65535}' ] }, url: 'https://example.org/' }

blakebyrnes commented 4 years ago

Would you mind showing me 1) your package.json (did you pull down version 1.0.0-alpha6?) and 2) what is in your node_modules\@Secret-agent\mitm\socket directory?

neryash commented 4 years ago

Package.json: { "name": "keadam", "version": "1.0.0", "description": "", "main": "app.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "", "license": "ISC", "dependencies": {} } node_modules@Secret-agent\mitm\socket: build.sh configure_tcp.go configure_tcp_opt_unix.go connect.go dialer_proxy.go emulate_tls.go go.sum clienthello_safari13.go configure_tcp_opt.go configure_tcp_opt_windows.go dialer.go domain_socket_piper.go go.mod