Open goranefbl opened 7 years ago
You'll need (at least) node to run scraperjs. If you wish to use the dynamic scraper you'll also need to have access to phantomjs. Cheers
On Fri, Jan 27, 2017 at 7:50 PM +0000, "Goran Jakovljevic" notifications@github.com wrote:
So i wrote a script thats working great on node, but I am looking into pushing it to a hosting where backend is PHP. Every day, cron would run a scraper script and get some data.
Can you point me to a direction where i could get more info on how to do this, and if its even possible.
Thanks for great script.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
Hello ruipgil,
I am also interested in running the dynamic scraper from php in production (for different urls and different users).
I could use exec() to start it with a command line but I have 2 problems from there:
1 - send the data in real time to a url containing Jquery (http://exqmple.com/my_widget.js) 2 - handle the end of the process once the user leave my_widget
You mentioned "have access to phantomjs" in your previous answer, what do you mean by that?
Can I do [1] and [2] with scraperjs only or do I need another script like socket?
Thanks in advance!
Hello again,
I have tried multiple things including detaching the process with screen
command but now I am considering using a chat with socket.io here and run the dynamic scraper inside. This will allow me to easily handle the end of the process i guess...
When the chat session is over the dynamic scraper should be ended.
For the moment it is just on paper, do you think it is a better approach?
How can I end the dynamic scraper?
So i wrote a script thats working great on node, but I am looking into pushing it to a hosting where backend is PHP. Every day, cron would run a scraper script and get some data.
Can you point me to a direction where i could get more info on how to do this, and if its even possible.
Thanks for great script.