search

chill117 / proxy-lists

Node.js module and CLI tool to get proxies from publicly available proxy lists.
MIT License
626 stars 106 forks source link

END Event is not called #90

Closed aorfevre closed 5 years ago

aorfevre commented 5 years ago

Hi,

The end event is not called on my side.

My option is :

  var ProxyLists = require('proxy-lists');
  var options = {
    sourcesBlackList:["coolproxy","proxy-list-org","freeproxylist","new-net-time","blackhatworld","kingproxies","bitproxies","sockslist","proxydb"]
  };
  gettingProxies.on('data', function(proxies) {
   });

  gettingProxies.on('error', function(error) {
    // Some error has occurred.
    console.error("ERROR",error);
  });
  gettingProxies.once('end', function() {
    // Done getting proxies.
        console.log("END");
  });

Moreover; I have that issue that is showing up 

ERROR { Error: unable to verify the first certificate
    at TLSSocket.onConnectSecure (_tls_wrap.js:1032:34)
    at emitNone (events.js:115:13)
    at TLSSocket.emit (events.js:218:7)
    at TLSSocket._finishInit (_tls_wrap.js:633:8) code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE' }

Thank you for your support,

Looking forward to provide you more info if necessary

mattwalter91 commented 5 years ago

Did you ever resolve this issue? I'm having the same problem...

aorfevre commented 5 years ago

No. I ended up having a setTimeout.

chill117 commented 5 years ago

This should be fixed now with the latest release (v1.18.0)

stylesuxx commented 5 years ago

@chill117 just upgraded to v1.18.0 still the end event does not seem to be emitted...

EDIT: it seems some of the proxy lists simply never resolve. Just run the tests locally. Almost none of the sources returns valid proxies (they time out after 30sec). I can access those sites fine manually - also the tests pass on travis.

chill117 commented 5 years ago

Re-opening this as it looks that some of the sources will hang forever if they have some error.

stylesuxx commented 5 years ago

For me this was fixed with https://github.com/chill117/data-sourcer/commit/1e0ddd937955a0066371096c4003430f218dff7e

chill117 commented 5 years ago

@stylesuxx

I've made significant improvements to the under-lying data-sourcer module - in particular the list-crawler abstract. I have not yet created a new release for proxy-lists module, but I will in the coming days.

Can you please checkout and test the latest proxy-lists master branch and let me know if the situation has improved for you?

stylesuxx commented 5 years ago

@chill117 I checked out proxy-lists master and data-sourcer looking good so far, but I still have this issue => https://github.com/chill117/data-sourcer/pull/16

But other than that, end signal is definitely emitted.

chill117 commented 5 years ago

Closing this issue as the end event is now sending properly. I've created a separate issue for cleaning up the browser tabs in the data-sourcer module: https://github.com/chill117/data-sourcer/issues/17