Error: connect ECONNREFUSED

Carr1005 / lighthouse-batch-parallel

A module for helping collecting websites' Lighthouse audit data in batches. Get the report data stream in CSV, JS Object or JSON format. Also provide a cli-tool to generate the report file in CSV or JSON format directly.

Apache License 2.0

28 stars 10 forks source link

Error: connect ECONNREFUSED #10

Open tidianelyahmed opened 3 years ago

tidianelyahmed commented 3 years ago

This is my first time working with lighthouse-batch-parallel and Node.js and I ran into this problem: I have this error on most of my urls, have you ever had this error?

Error: connect ECONNREFUSED 127.0.0.1:39855 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) { errno: 'ECONNREFUSED', code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 39855 } Device = desktop || URL = https://www.vdm-reya.com/fusees Total: 200 || Remain: 183 Error: connect ECONNREFUSED 127.0.0.1:37451 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) { errno: 'ECONNREFUSED', code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 37451 } Error: connect ECONNREFUSED 127.0.0.1:33513 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) { errno: 'ECONNREFUSED', code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 33513 }

Carr1005 commented 3 years ago

@tidianelyahmed , could you provide your code and input URLs 😃? Let's see what can we find!

tidianelyahmed commented 3 years ago

Thank you for your reply.

Above you will see my CLI and the header of my csv file(URL):

lighthouse-batch-parallel -n 64 -l input.csv

Carr1005 commented 3 years ago

Hi~ @tidianelyahmed, I tested it at my local then I got the same error messages from URLs below. I collected them and tested them separately, but it ends up having no problem. Then I found all of these URLs are after the URLs which are actually PDF files, so I think that's the main problem, the browser instance cannot handle it properly and causes problems when it's reused for next process.

https://m.piecesauto-pro.fr/pieces-detachees/filtre-a-huile-hydraulique https://www.yakarouler.com/pieces-auto/m/renault https://www.calameo.com/books/001104130b9472ba69bac https://www.mabeo-direct.com/A-555290-total-huile-hydraulique-azolla-zs-46-20l https://fr.shopping.rakuten.com/s/jouet+betonniere https://www.groupeselectrogenes.fr/portfolio-item/ats-switch

I am not sure if you are intentional to test them, but actually, if you run the Lighthouse in the Devtools to monitor those PDF files, you also get a lot of errors. I think PDF reader on Chrome has different behavior in comparison to rendering a web page! Just for your information, maybe you would like to get rid of those PDF files first?😅

tidianelyahmed commented 3 years ago

Hi, thank you for your reply. Indeed I deleted all the urls containing "pdf" and I sent 100 urls, the last 20 urls of input.csv had no errors.I redid for 1000 urls about 10% were OK, and these are the last urls of the table input.csv. I'm really lost. All bugs are about errno: 'ECONNREFUSED'.

Carr1005 commented 3 years ago

@tidianelyahmed, sorry that I don't really get it, so do you mean after you get rid all of PDF URLs, then you put 1000 URLs in the input.csv, but only 10% of them are successfully scanned? So have you tried what I've done above, maybe collect the failure cases and audit them separately? Or check the URL before the failure cases?

tidianelyahmed commented 3 years ago

I will test that. I also found that if I reduce the number of workers from 64 to 10 it works (90% of the URLS are Ok). I think the ports have something to do with it too. A month ago the cli:lighthouse-batch-parallel -n 64 -l input.csv did not give any error.If it persists I will reinstall it on another VM( i am in gcp). Thanks, I'll keep you informed.

Carr1005 commented 3 years ago

Thanks, @tidianelyahmed! Sorry that I don't have a GCP environment. If you find out any clue, please update more info here!