Scripts stop on disconnected VM - pending some user checks

[tejas-skopiq]

Hi Ken, We are using TagUI to automate one of the use case where we want to launch chrome browser -> enter url -> perform login -> launch Citrix app.

We are executing this on windows VM machine. When I log into machine and execute scripts, its working fine. However, when I jump off from RDP (VM) with below script where session remain active and it also render visual. My scripts also executed, it able to launch chrome -> enter url and then stopped.

Any idea or suggestions or pointer for the above issue?

Regards, Tejas

[kensoh]

Hi Tejas, can you paste the log files here to see if there are more clues what is going on. The useful log files will be the log file for your script, the tagui\src\tagui_chrome.log, and if you are using visual automation, then the logs in tagui\src\tagui.sikuli folder.

There might be some behaviour that happens when the VM is disconnected that results in this outcome, but there's little other reference of this issue and I hear that some users are able to run VM in background without issues.

[tejas-skopiq]

Hi Ken, Attached logs and also script for the ref.

I am executing this command on VM, this will kick me off from the RDP but keep the session active and it also keep rendering display.

for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do ( %windir%\System32\tscon.exe %%s /dest:console )

tagui_files.zip

[tejas-skopiq]

Hi Ken, Did you get chance to look into this?

Regards, Tejas

[kensoh]

From the SikuliX log, below is the error. It looks like you have OpenCV on your VM. Try running this on a VM without OpenCV installed to see if the error happens. It may be a conflict between the OpenCV on your computer and the one used by SikuliX.

If after uninstalling your OpenCV or using another VM without OpenCV, it might be the display is somehow no longer rendered for SikuliX computer vision engine to consume and lead to the error. I'm afraid I do not have further details on SikuliX being used on a VM that is running in the background to advise further.

[tagui] ACTION - present C:/TagUi_app/tagui/src/chrome_is_max.png
[error] script [ tagui ] stopped with error at line --unknown--
[error] Error caused by: Traceback (most recent call last):
  File "C:\TagUi_app\tagui\src\tagui.sikuli\tagui.py", line 417, in <module>
    intent_result_value = parse_intent(tagui_intent)
  File "C:\TagUi_app\tagui\src\tagui.sikuli\tagui.py", line 369, in parse_intent
    return visible_intent(script_line);
  File "C:\TagUi_app\tagui\src\tagui.sikuli\tagui.py", line 300, in visible_intent
    if exists(params):
]
    at org.opencv.imgproc.Imgproc.matchTemplate_1(Native Method)
    at org.opencv.imgproc.Imgproc.matchTemplate(Imgproc.java:2612)
    at org.sikuli.script.Finder$Finder2.doFindMatch(Finder.java:898)
    at org.sikuli.script.Finder$Finder2.doFind(Finder.java:868)
    at org.sikuli.script.Finder$Finder2.find(Finder.java:617)
    at org.sikuli.script.Finder.find(Finder.java:280)
    at org.sikuli.script.Region.runFinder(Region.java:2958)
    at org.sikuli.script.Region.doFind(Region.java:2894)
    at org.sikuli.script.Region.access$300(Region.java:23)
    at org.sikuli.script.Region$RepeatableFind.run(Region.java:3175)
    at org.sikuli.script.Region$Repeatable.repeat(Region.java:3126)
    at org.sikuli.script.Region.exists(Region.java:2442)
    at org.sikuli.script.Region.exists(Region.java:2466)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
org.opencv.core.CvException: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.2) C:\build\3_4_winpack-bindings-win64-vc14-static\opencv\modules\imgproc\src\templmatch.cpp:1107: error: (-215:Assertion failed) _img.size().height <= _templ.size().height && _img.size().width <= _templ.size().width in function 'cv::matchTemplate'
]

[tejas-skopiq]

Hi Ken, Based on your input, I have checked VM machine... but I did not find any opencv installed... attached few snapshots .... am i missing something here?

[kensoh]

org.opencv.core.CvException: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.2) C:\build\3_4_winpack-bindings-win64-vc14-static\opencv\modules\imgproc\src\templmatch.cpp:1107: error: (-215:Assertion failed) _img.size().height <= _templ.size().height && _img.size().width <= _templ.size().width in function 'cv::matchTemplate'
]

Oh Tejas, it is on the last line of the Sikuli log, see above. The error is pointing to this OpenCV file below. This is not the way SikuliX uses OpenCV as it has its own OpenCV packaged within itself. So not sure why there is below OpenCV, which may be the cause of error.

OpenCV(3.4.2) C:\build\3_4_winpack-bindings-win64-vc14-static\opencv\modules\imgproc\src\templmatch.cpp

Uninstalling or removing may still not fix the issue, because SikuliX I don't think is tested to work for VMs running in background mode.

[tejas-skopiq]

Right Ken... even When I am running 4-5 times, success ratio is 70%.... 30% it fails... and for automation, I cant keep VM to be logged-in 24x7, that's also biggest issue... Anyways, thank you so much Ken for your time and share feedback....

Do you suggest any other similar tool which can work on VM which is in background?

Regards, Tejas

[kensoh]

I haven't heard of other tools that are designed to run on VMs and in the background. Your best bet will be trying some community free versions of commercial tools like UiPath or Automation Anywhere. If those enterprise tools can't support the use case running in VM in background, it is unlikely other open-source tools have the capability to meet this uncommon use case.

If the reason for not running 24 x 7 is the cost, then maybe a workaround is automate / schedule the logging in and out so that during the times when automation is needed VM will load, and after that VM will close. Microsoft is partnering with Automation Anywhere and UiPath to have their apps running on Azure cloud servers, maybe they'll have a solution that works for this use case.

[kensoh]

Closing issue for now pending further new inputs!