Steveorevo / node-red-contrib-nbrowser

Provides a virtual web browser (a.k.a. "headless browser") appearing as a node.
35 stars 13 forks source link

interrupted browser behavior when running 2 flows concurrently #9

Closed chuyaguo2014 closed 6 years ago

chuyaguo2014 commented 6 years ago

I am trying to use nbrowser to crawl some websites and I realized that, if I run 1 crawl at a time, everything works; but if I kick off the 1st crawl and then the 2nd one shortly afterwards, looks like the browser immediately starts the 2nd crawl and completely aborts everything it was doing in the first crawl.

Looks like the browser instance is somewhere shared across the flows - Is this an intended functionality of nbrowser where only 1 flow is expected to use the nbrowser node at a time? Or am I missing something obvious?

Here's how you can reproduce the issue: I made 2 gists from the flows - they are crawling the same website except that:

(please let me know if you have issue running the flows in the gist. Thank you!)

Steveorevo commented 6 years ago

If you would like to use more than one instance, did you try changing the instance name in the second field from msg.nbrowser to msg.nbrowser2? In theory, you could can store the window instance name in a custom variable name of your choosing. Just be sure that any subsequent nbrowser nodes reference it to act upon it; most importantly the close operation to free the variable.

chuyaguo2014 commented 6 years ago

It worked! Thank you so much! :)

chuyaguo2014 commented 6 years ago

@Steveorevo oops sorry one more question I forgot to mention - Even though "I can store the window instance name in a custom variable name", within the same flow, I would still only have 1 window instance, right?

Here is why I'm asking:

I'm thinking about wrapping some of the common browser behaviours into a subflow (something like "go to a page, wait for a selector to appear, click it, save the HTML, and go back"). And I plan to use the nbrowser node inside of this subflow.

And outside of the subflow, I would give it different input (for the start URL, for example), and kick off the crawls.

Here's an example: image

Even though I can store the window instance name like msg.subflow_nbrowser, there will still only be 1 place where I set it (i.e. inside the subflow when I first initialize the nbrowser node) - does this mean, given the subflow design, I can still only run 1 crawl at a time?

I know I can kick off multiple crawls in multiple windows if I don't use subflows. I was just wondering if this is also possible with subflow. (my intuition is telling me "no" but I am not an expert in this area so I thought I'd ask). :)

Thank you so much!

Steveorevo commented 6 years ago

Depends how you create the instances along a wire. If you created unique instances outside of the subflow; the subflow could act on the "nbrowser" instance internally (three times).

screen shot 2018-02-19 at 1 29 19 pm
chuyaguo2014 commented 6 years ago

oh I see! that makes a lot of sense! Thank you so much!