dimkir
commented
6 years ago Hey Matt, glad that Nightmare works for you on Lambda. I will start with your original question
I'm looking to see if there's a way to view, and more importantly, interact with the Electron browser display when running Nightmare automation on Lambda.
Short answer - NO.
Long answer - YES (but it would be impractical)
Basically Electron runs within virtual X-session on Lambda. In theory user can connect to the X-session remotely and do all the manipulations and stuff. Nowadays it is a common practice to work on workstations remotely.
However Lambda has a lot of limitations. One of them prohibits incoming TCP/IP connections, which would require "jump hosts", "port forwarding" and a lot of wizardry. This wizardry most likely will work, but I have a feeling it would be brittle and hard to debug. So it is unlikely of practical use for wide audience.
Enter human
Your question is about what you call "semi-automated Nightmare on Lambda". For the sake of explanation, I would prefer to call it "HUman Assisted Nightmare", which highlights important role of human involvement. And that human involvement will drastically slow dow n Nightmare sessions.
A conventional Nightmare on Lambda simulation would have no problems running 1000 sessions in parallel. Because those sessions do not require human assistance - Nightmare on Lambda provides huge benefit of scale and simplicity.
Bringing in human, changes balance - we have bottleneck of speed (eg. humans need to process "captchas").
- By involving human we can lose an order of magnitude of productivity (eg. process 100 sites per/hour against 1,000) and we lose the benefit of scale.
- The wizardry required to make things work makes us lose second benfit - simplicity.
HUman Assisted Nightmare seems to be a better fit for conventional servers.
Managing user interaction lifecycle
So now you can setup a regular server with X-sessions on it, provide access to users and start running your sessions. New challenge will emerge: your total session may take 60 seconds, but you want to involve human only for 10 seconds to key in captcha. Now you have to manage interaction lifecycle. This challenge was there since the very beginning (since considering Lambda for the case), but even on regular server this would be tricky (but on Lambda it would be much more annoying).
So what would be the "simplest" setup for your case:
- Run server with X-session and ability to remote connect
- Allow your "user" tp connect to the session remotely
- Run your NightmareJS scripts (arranged in a way which will show browser only when interaction is needed)
- After "captcha" input done - hide the browser and continue remaining 50seconds of session
- Immediately show new window with new session; goto 4
ps. I know this may not be the cool and serverless way to do things, but it has the simplicity factor and you can test interaction on local machine and then move it to remote servers. I guess you thought of it before yourselves. But I just wanted to outline my perspective to help somehow. However it is always open for discussion.
themattrobinson
I'm looking to see if there's a way to view, and more importantly, interact with the Electron browser display when running Nightmare automation on Lambda.
With this excellent tutorial as my guide, I've successfully set up multiple Nightmare automation jobs on Lambda.
While full automation is definitely the ideal, there are times when a real human must interact with a given web page to move past a certain point- i.e. a Captcha form that must be completed.
I have dreamed up this "semi-automated" approach that would basically "nightmare.wait()" for the User to interact with the browser, then return to full automation afterwards.
A high-level example would be something like this:
I have already implemented this locally during testing (setting Nightmare config option "show" to true, then clicking on parts of the page shown while it "waits" for the next automated step), so I know the semi-automatic aspect of it works.
However, the part I'm unsure about is how to replicate this when the Nightmare automation is running on Lambda. Is there any way for me to show the Electron browser window and interact with it when its on Lambda? If not, if I put the Nightmare automation on my own web server, would there be a way for me to view and interact with the Electron browser window through an iframe in a real web browser (or some other similar way)?
I haven't seen a single example, or even mention, of anything like this on the web yet, but being able to achieve this would greatly enhance the capabilities of web automation. Getting around "bot blockers" and other anti-automation roadblocks would be much more feasible when it truly thinks a human is interacting with it (which it would be) than straight automation.