phantombuster / nickjs

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
https://nickjs.org
ISC License
500 stars 48 forks source link

Heroku: Error: connect ECONNREFUSED 127.0.0.1:9222 #47

Open wbelk opened 5 years ago

wbelk commented 5 years ago

could not connect to chrome debugger after 24 tries (10s): Error: connect ECONNREFUSED 127.0.0.1:9222

this just popped up for some reason on heroku, i have legacy dynos running the same code and working perfectly. tried to add a fresh dyno and getting this error for the first time

settings are: Stack: heroku-16 node: 8.9.0 npm: 5.5.1

msaus commented 5 years ago

@wbelk possibly, it is similar to this? Cannot connect to the target: connect ECONNREFUSED 127.0.0.1:9222

msaus commented 5 years ago

Well...when I deploy to production environment I am getting the same error. There are no google-chrome instances running and so I have no idea what to do next.

node: v8.12.0 npm: 6.4.1 google-chrome: 69.0.3497.100

msaus commented 5 years ago

I figure this out, but, this is really weird for me.

Before, start taking about how I figure this out, nodejs and pm2 are installed globally. The reason why I have pm2 is that I used to have nodejs based API with nickjs.

■Steps to figure this out.

① Execute my old nodejs based API with 1 process. ② Create HTTP request to API from 1. Note: nickjs should be used by the API ③The process from ① should be kept all the time in order to run nickjs, otherwise, I get the same error.

I do not know why I have to keep running my old API. Also, I tried restaill nodejs, pm2 and Google Chrome. But, this dose not work.

Of course, when I get this error, chrome is not up and running.

If you have any idea, please let me know.

paps commented 5 years ago

If I understand correctly, your NickJS script, for some reason, cannot start the chrome subprocess in your environment. But in older Heroku environments, it does work like it should.

Can you please add debug: true to your NickJS' constructor options? And report back with what it shows

In containerized/dockerized environments, starting chrome can be a pain. A simple google search shows many people having difficulties. For example chrome must be started with --no-sandbox (which NickJS does), the container should have a big shm size, etc... Maybe the newer Heroku has stricter container settings?

elhenro commented 5 years ago

I have the same error on Ubuntu 16.04, I tried node v8 and v11.6, chromium, chrome 70, 71,72,73(unstable).. I specified export CHROME_PATH="/usr/bin/google-chrome"; , also tried /opt/google/chrome/...

Edit: I solved it by starting chrome 71 like this: google-chrome --remote-debugging-port=9222 --no-first-run, --no-default-browser-check, and --user-data-dir

paps commented 5 years ago

@elhenro are you starting chrome by hand alongside NickJS?

elhenro commented 5 years ago

affirmative, the only way for me right now is to quit chrome completely and then start it with those options

paps commented 5 years ago

Can you please add debug: true to your NickJS' constructor options? And report back with what it shows? (do not start chrome by hand, I just want to see what happens on NickJS' side when it fails)

msaus commented 5 years ago

Sorry for late reply. Currently, my app is running on production and I can not stop it right now. So, when I have a chance for it, I will add debug: ture to see what happen to Nickjs. Regards.

bbathel12 commented 5 years ago

I'm having the same issue on windows ubuntu wsl here's the output of the debug CHROME STDOUT: Starting ChromeDriver 74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}) on port 9515 Only local connections are allowed. Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code. ERROR: could not connect to chrome debugger after 8 tries (10s): Error: connect ECONNREFUSED 127.0.0.1:9222

msaus commented 5 years ago

@bbathel12 I guess you have to give the permission to Windows.

msaus commented 5 years ago

@paps Ubuntu 16.04.5 LTS

I got following errors.

CHROME STDERR: [0412/123550.772631:ERROR:gpu_process_transport_factory.cc(980)] Lost UI shared context. CHROME STDERR: DevTools listening on ws://127.0.0.1:9222/devtools/browser/935baa40-9193-42d8-8709-06b781803ec7 Something went wrong: could not connect to chrome debugger after 24 tries (10s): Error: connect ECONNREFUSED ::1:9222

msaus commented 5 years ago

I am still getting the same error and so I have to do the follow the steps that I mentioned. By the way, I am getting error on Ubuntu machine and I do not know what to do next. If you can help me out, I will give you more details.

paps commented 5 years ago

It might be because it's a too recent version of Chrome. Can you try with Chrome 65? I know it's old but it might do the trick

msaus commented 5 years ago

@paps when I have a chance for that, I will switch to chrome 65. Can it downloaded from here? https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/chromium-browser/65.0.3325.181-0ubuntu0.16.04.1/chromium-browser_65.0.3325.181.orig.tar.xz

msaus commented 5 years ago

Then, I guess you had better to write what chrome version supported, otherwise, people get stacked like me.

paps commented 5 years ago

I'll update the readme if you can confirm that it's working with an older version :)