search

phantombuster / nickjs

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
https://nickjs.org
ISC License
500 stars 48 forks source link

Scaling #31

Closed dalepo closed 6 years ago

dalepo commented 6 years ago

How is scaling supposed to work with nickjs? Does instantiating a new nickjs starts a new session (suppose you want to keep tabs independent from each other)

jprrabanal commented 6 years ago

i think the headless chrome does not support multiple tabs?

paps commented 6 years ago

Headless Chrome supports multiple tabs, and that's one way of "scaling" it. nick.newTab() opens a new tab. However tabs as not isolated in the sense that they share cookies, cache, local storage and so on. So this is not a solution that works for all cases.

Another way would be to completely isolate Chrome instances in containers (Docker or similar). That's what we do on our NickJS-as-a-service platform, Phantombuster.com.

NickJS does not support being launched multiple times because it starts a Chrome process each time, in the exact same configuration. A fix for this would be to add a launch option where you can specify a Chrome port, profile and other Chrome launch options. I'll get to it one day, that's for sure. In the meantime, PRs welcome! :)

jprrabanal commented 6 years ago

@paps Thanks for the enlightenment!

msaus commented 5 years ago

excuse me for cutting in. Is the Chrome launch options out or still under development? thanks