search

open-rpa / node-red-contrib-tagui

Mozilla Public License 2.0
8 stars 4 forks source link

Steps to run TagUI RPA on OpenFlow Cloud #7

Closed kensoh closed 2 years ago

kensoh commented 2 years ago

_See a video demo of these steps, all the steps below should take under 5 minutes_

Create OpenFlow Account

  1. visit https://app.openiap.io/#/Login
  2. login using Microsoft ID or Google ID

Create Node-Red Instance

  1. click NodeRed on the top menu
  2. select Latest TagUI Nodered option
  3. click Create Nodered to create instance
  4. wait for some time, click Open Nodered

Login to Node-RED on OpenFlow Cloud

  1. URL goes to https://xxx.app.openiap.io
  2. click Sign in with SAML button

Create an Example TagUI RPA Workflow

  1. install node-red-contrib-tagui from Manage Palette     (top-right hamburger menu --> Manage Palette --> Install)

  2. install node-red-contrib-image-output from Manage Palette     (top-right hamburger menu --> Manage Palette --> Install)

  3. import example workflow for node-red-contrib-tagui     (top-right hamburger menu --> Import --> Examples)

Deploy and Run TagUI RPA Workflow

  1. click Deploy button at top-right of menu
  2. click button to trigger cute kitten input

Spinning down the Node-RED instance

  1. after you are done, click Delete Nodered in OpenFlow
  2. if this is not done, the instance will be terminated after 24h
  3. more info on paid plans (persistent running, more ram etc)

PS - This issue here was originally created to troubleshoot some rough edges with running TagUI on OpenFlow's Node-RED. The issues are now resolved, you can ignore the exchange between Allan and me and technical details below.

kensoh commented 2 years ago

PS - no hurry, only when you have time to look into this. Separately, I've replied your Telegram message on the Python issue

skadefro commented 2 years ago

I wrote on telegram for hints/help on what to look for to troubleshoot tagui :-) ... There is no errors in the console, and if i call tagui from a commandline inside docker it just do the same ( nothing and never returns ) ... Do you have an idear on something I can look at ?

kensoh commented 2 years ago

Below was my replies on Telegram. I see.. If running TagUI nothing happens other than Chrome opening a blank window, it could be for some reason unable to connect to Chrome. No curl command available, Chrome's websocket not allowed etc.

Thanks for your inputs! let me try replicating and debugging from a Docker instance on my Mac laptop.

TagUI and the Python rpa package should work across different Python versions whether 2.X or 3.X. lemme think what can be done to troubleshoot this. Does the problem happens for TagUI human language version or the Python rpa version? or both

If it is the Python version only,

  1. Try this below to see where TagUI is installed -

import rpa as r r.tagui_location()

  1. Assuming it is at /home/user folder, then try this from terminal -

/home/user/.tagui/src/samples/1_yahoo chrome

To see if any other error messages can show up

For Linux, TagUI will require curl command to be available. A possible cause might be curl command is no longer available. curl is needed by TagUI and the rpa package to search for the websocket connections at Chrome, so that TagUI can connect to it

skadefro commented 2 years ago

Curl is present. Wget too, that is what the image uses to get the zip files. I have not tested python version. The nodered node uses the real tagui in live mode. I just add the python version for those who also want to use that instead.

kensoh commented 2 years ago

Thanks Allan, I tried running below locally on my Mac and got the following cryptic errors when trying to run Chrome. Even with no parameters Chrome can't seem to launch successfully. The image used is https://hub.docker.com/r/openiap/nodered-tagui and I'm connecting to the terminal of the nice_leakey container.

$ cd /home/openiapuser/tagui/src

$ google-chrome --user-data-dir="/home/openiapuser/tagui/src/chrome/tagui_user_profile" --remote-debugging-port=9222 about:blank --headless --disable-gpu
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
Failed to generate minidump.Illegal instruction

$ google-chrome
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
[1215/041308.934712:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)
[1215/041308.934905:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq: No such file or directory (2)
Trace/breakpoint trap
$ [39199:39199:0100/000000.958178:ERROR:zygote_linux.cc(646)] write: Broken pipe (32)

\ Adding on, in the same folder, when I try to run without a browser (ie using PhantomJS headless browser), TagUI is able to run, suggesting the issue might be something around Chrome browser. Any thoughts on these findings?

$ ./tagui ../flows/samples/1_google.tag -nobrowser

START - automation started - Wed Dec 15 2021 04:20:56 GMT+0000 (UTC)

https://www.google.com - Google
type //*[@name="q"] as latest movies[enter]
Docker Hub
kensoh commented 2 years ago

Some links on this error message. Some approaches from Docker angle, some from Chrome angle.

https://github.com/jessfraz/dockerfiles/issues/350 https://github.com/jessfraz/dockerfiles/issues/65#issuecomment-266532289 https://forums.docker.com/t/docker-user-namespaces/16885/3 https://coderedirect.com/questions/542850/google-chrome-failed-to-move-to-new-namespace

When I tried below by adding --no-sandbox flag, the Chrome command runs. I remember for my Colab notebook, I have to run with --no-sandbox in order to launch Chrome as the root user.

Could it be for some reason, previously the image is not running as the root user but is now executed as the root user? This train of thought is puzzling because whoami command shows user as openiapuser and not root.

$ google-chrome --user-data-dir="/home/openiapuser/tagui/src/chrome/tagui_user_profile" --remote-debugging-port=9222 about:blank --headless --disable-gpu --no-sandbox
Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
[1215/082243.777199:ERROR:bus.cc(392)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory

DevTools listening on ws://127.0.0.1:9222/devtools/browser/8a560d95-41cc-4f50-a535-2ec4bcd8fe1d
[1215/082243.806001:ERROR:gpu_init.cc(453)] Passthrough is not supported, GL is swiftshader, ANGLE is
Docker Community Forums
Docker user namespaces
Looking at the thread on Jess’s GH repo it seems that Chrome wants to create its own user namespace sandbox which is not supported by some kernels. Usually capabilities refer to things like CAP_NET_ADMIN which define privileges of individual processes, e.g. to “spy” on the network traffic. You might be thinking of kernel configuration options, which is what’s needed here and would require a kernel compile and re-boot. I’d be surprised to find out that user namespace remap would help the probl...
skadefro commented 2 years ago

Could this be part of the solution ? If i use the chrome.json trick it works. If I add --no-sandbox --headless it also works. ( I get "DevTools listening on ws://127.0.0.1:9222/devtools/browser/6e93d698-cfa6-4d78-a007-1f9a3c4b7a1a" ) Not sure how to "inject" that into tagui to test. With chrome.json set using security_opt: in docker, I instantly get no result (and no error) in tagui.

Using chrome.json should be more secure than using sandboxing ( not sure why ? ) but if that is the solution i would recomend we inject it into the image, and make some option to tagui to tell it to use this file. I do not want to map it externally, is going to be a nightmare to support people on docker and kubernetes on how to add that file, so would be better if tagui could somehow detect the file or we can tell it somehow to use that parameter. Or we could go for the easy solution and just use --no-sandbox --headless ? what do you think ?

skadefro commented 2 years ago

When I tried below by adding --no-sandbox flag, the Chrome command runs. I remember for my Colab notebook, I have to run with --no-sandbox in order to launch Chrome as the root user.

Could it be for some reason, previously the image is not running as the root user but is now executed as the root user? This train of thought is puzzling because whoami command shows user as openiapuser and not root.

In docker, you by default run as root. But in my dockerfile's I revert to an normal user, and is not running as root ( that is also why you used /home/openiapuser/tagui/src )

kensoh commented 2 years ago

Hi Allan, I've booked your time on Friday 11am your time to troubleshoot this. In the meantime, that's interesting findings, the link that you shared - https://stackoverflow.com/a/64460373/17318661

Do you know what happened between previously when the image works and now? I'm trying to weigh what would be the best approach. From TagUI's perspective, using both chrome.json or --no-sandbox method requires the same hacks (modifying the tagui/src/tagui file).

However, from TagUI's perspectively only, using --no-sandbox is the preferred method. This is because for at least 6 months, when I created the Google Colab versions of TagUI, --no-sandbox method has been used successfully. Thus, from usability perspective this method is most likely to cause less problems or edge cases failures for users.

However, if we can confirm from Docker container perspective, using chrome.json is the best practice, then that should be considered as the approach to take. However, that may invite user support issues in the meantime because that path for TagUI has not been 'walked through' by big enough number of users.

The solution to fix (what I did for the Colab notebook) is as part of the initialisation, make the modifications required in tagui/src/tagui file, so that either --no-sandbox is used or chrome.json is used by default. But in this case, it is tricky because there is the update option from Node-RED TagUI module. Any changes will be overwritten when that happens.

On the other hand, to have --no-sandbox or chrome.json code as part of TagUI repository will be hard to implement. It'll be hard to accurately automatically detect whether this should be done during execution, and if the 'diagnosis' to use these flags are wrong, it can open up errors for the main usage scenario of running locally.

Stack Overflow
google-chrome Failed to move to new namespace
Im trying to run google-chrome --headless inside a docker container as a non-root user to execute some tests. Everytime Im trying to start it, it throws following error: google-chrome --headless ...
skadefro commented 2 years ago

It's not better to use chrome.json in docker, it was just an alternative to using --no-sandbox ... if that works for you, then lets keep with that. But I just noticed something. If i run google-chrome --user-data-dir="/home/openiapuser/tagui/src/chrome/tagui_user_profile" --remote-debugging-port=9222 about:blank --headless --disable-gpu --no-sandbox then i can connect to chrome no problem, tested using 'curl -s localhost:9222/json' but if I run '/home/openiapuser/tagui/src/tagui robot.tag -headless "kitten"' i get this in the console /home/openiapuser/tagui/src/tagui: line 395: 29510 Illegal instruction $chrome_command --user-data-dir="$TAGUI_DIR/chrome/tagui_user_profile" $chrome_switches $window_size $headless_switch > /dev/null 2>&1 Is that a problem ? is this something that needs to be addressed ?

kensoh commented 2 years ago

Hi @skadefro yes this is a problem. TagUI in the execution script, does not have --no-sandbox. It has to be introduced in some way. For the Colab notebook examples, I introduced them by modifying the tagui/src/tagui file on initialisation of the cloud instance. But in this case, it is hard. Because any hack done will be gone when users choose the update checkbox in Node-RED. Unless as part of the update logic in Node-RED module, the modification is made post-update.

Another solution would be introducing a new option for TagUI, for example -root. And when this option is detected, TagUI will invoke Chrome with --no-sandbox option. Upside would be cleaner implementation on the Docker image (will still require doing an update first to get the latest patched version). Downside would be for some reason a low risk of breaking existing use cases, since the part of the execution logic is flowed through for all users during initialisation.

kensoh commented 2 years ago

Am chewing more on this... Catch you for the call later?

skadefro commented 2 years ago

Yup, we have a meeting in 2½ hour .. :-)

kensoh commented 2 years ago

Enhancement added in https://github.com/kelaberetiv/TagUI/issues/1151 in TagUI v6.102 -

By creating a dummy file (either blank or with text it's both ok) tagui/src/tagui_no_sandbox, TagUI will run Chrome with the --no-sandbox option. This option is needed to run Chrome as root user (for eg on the cloud) or in a Docker container.

kensoh commented 2 years ago

Done!

Posted on LinkedIn a video demo - https://www.linkedin.com/posts/kensoh_see-how-you-can-run-tagui-free-rpa-on-the-activity-6881841265084915712-mBeO/

Showcasing this on TagUI GitHub - https://github.com/kelaberetiv/TagUI#do-rpa-anyway-you-want

Showcasing this on TagUI usage guide - https://tagui.readthedocs.io/en/latest/advanced.html#running-tagui-on-the-cloud

Ken Soh on LinkedIn: See how you can run TagUI free RPA on the cloud, using OpenFlow by
See how you can run TagUI free RPA on the cloud, using OpenFlow by Allan Zimmermann. In under 5 minutes, you can get a TagUI workflow running from your...
GitHub
GitHub - kelaberetiv/TagUI: Free RPA tool by AI Singapore
Free RPA tool by AI Singapore. Contribute to kelaberetiv/TagUI development by creating an account on GitHub.
Advanced concepts — TagUI 6.46.0 documentation
marcelocecin commented 1 year ago

Hi Ken, how are you? I miss more complete examples for NodeRED Is it possible to also use visual automation on the platform? thanks!

skadefro commented 1 year ago

NodeRed can run as the user, or as a service. On Windows that means with access to the desktop and UI or not. If you have a NodeRED instance running as your own user, you are logged in, the desktop is not locked, then you can do anything that your would normally do from TagUI including automating using image recognition. If how ever you are running the NodeRED as a service ( like pm2 ) or in docker, then you are limited to automating things that does not require an UI ( like browser automation ) hence the example is about doing browser automation. But in the tight context anything is possible.

kensoh commented 1 year ago

Hi @marcelocecin good good, just overwhelmed with work and family commitments, and settling in. I just logged on to GitHub to see if there is any reported bugs on rpa package and saw your exchange above with @skadefro. Hope all is good with you. Been out of action from open-source community as I try to keep up with work and family.