search

amoilanen / js-crawler

Web crawler for Node.JS
MIT License
253 stars 55 forks source link

Pair js-crawler with PhantomJS #49

Open Tynael opened 7 years ago

Tynael commented 7 years ago

Hey,

First of all I would like express my gratitude for developing this sweet web crawler.

Is there a way integrate js-crawler with PhantomJS?

I really need their functionalities in a single place as I would like to use PhantomJS for network monitoring.

Looking forward to hearing from you!

amoilanen commented 7 years ago

Hi Tynael,

Now js-crawler is using Node.js 'request' module to make requests. In the future versions it is possible to add support to using other request 'engines' such as PhantomJS.

Is there a way integrate js-crawler with PhantomJS? So in the future there are such plans, unfortunately no at the moment.

Please, note, that it seems that the active development of PhantomJS has ended, and a better option to use may be to use headless Chrome.

https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE

Thank you for using the crawler and asking a good question.