Add auto-screenshot of webpage or screen if element is missing - done

[marcelocecin]

hello ! is it possible to take a snap of the screen when an error occurs? as I programmed some cronjobs to run automatically, I would like to go beyond the error LOG, the screen print would help to better understand why the image was not found on the screen thank you !

[kensoh]

Hi Marcelo, thanks for your suggestion!

This is a great idea and I've started implementing it into TagUI.

Below video for info only, just left cleaning up and can commit - https://www.youtube.com/watch?v=kNqU2xPJOYs

Usually, such situations can be handled by doing a check like below -

if exist('submit_button')
    click submit_button
else
    // screenshot of webpage
    snap page to abc.png

    // screenshot of whole screen
    snap page.png to abc.png

It also be addressed by increasing wait time before TagUI times out, eg of various ways to use timeout step -

timeout 60
timeout 60 seconds
timeout 60s
timeout 60 sec
timeout 6.5 seconds
timeout 2.5

But nothing is as user-friendly as auto-capturing a screenshot, so we'll add it in. Will update soon.

[kensoh]

Above commit adds this feature now to TagUI. Users can download the latest copy of TagUI from here and unzip to overwrite your existing installation (please drag the folders under tagui\src to overwrite your existing installation) -

https://github.com/kelaberetiv/TagUI/archive/master.zip

In the next release, this auto-screenshot feature will become part of the packaged zip files.

Thanks Marcelo! Let us know how it goes and if any problem.

CC @ruthtxh

---

Video for techie's reference of evaluating and adding this great suggestion into TagUI -

https://www.youtube.com/watch?v=kNqU2xPJOYs

Am happy to be able to add this interesting feature within 24h of Marcelo's suggestion. (including testing of the new feature and existing features for regression problems)

[marcelocecin]

@kensoh you are the man! soon I will start some tests was thinking, would it be possible to send the error text to a webhook? it would also be great in case of screen capture, also send the image in base64 format thank you very much !

[kensoh]

Hi Marcelo, consider the following dummy webhook.

First, create a custom webhook function by defining your custom function in tagui_global.js and put it in your tagui/src directory. This is available to all flows. If you create tagui_local.js and put in your flow folder, it is available to all flows in that folder. Next, insert in tagui/src/tagui_parse.php, the webhook call, see working example below for the error message for missing web element (end_tx function) and the one for missing image element (call_sikuli function).

Running below flow will trigger the webhook call to do a REST API call and return response back. However, this would be a feature not used by most users now, so we won't add this into the feature set. For Base64, I think need to figure out the JavaScript code that works in the PhantomJS JavaScript engine, or hacking the chrome.capture() function in tagui_header.js to return the Base64 string for web automation use case and figure out how to convert the binary SikuliX screenshot to Base64 for the desktop visual automation use case. But I don't even have the solution myself lol.

https://google.com

// dummy api step to allow cross-domain traffic
api https://jsonplaceholder.typicode.com/users

click abcdefwxyz

tagui_global.js

function error_webhook(text_input) {
    casper.echo('[WEBHOOK TEST MESSAGE]\n' + text_input + '\n');
    call_api('https://jsonplaceholder.typicode.com/users');
    casper.echo('[REST API RESPONSE]' + api_result + '\n');
    try {api_json = JSON.parse(api_result);} catch(e) {api_json = JSON.parse('null');}
    casper.echo(api_result);
}

tagui_parse.php

function end_tx($locator) { // helper function to return ending string for handling locators
if ($GLOBALS['inside_while_loop'] == 0)
return "},\nfunction timeout() {this.capture('".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');".

"error_webhook('123')".

"\nthis.echo('ERROR - cannot find ".$locator."').exit(); this.wait(0);});}"."});".end_fi()."\n\n";
else if ($GLOBALS['inside_code_block']==0)
{$GLOBALS['inside_while_loop'] = 0; // reset inside_while_loop if not inside block
$GLOBALS['for_loop_tracker'] = ""; // reset for_loop_tracker if not inside block
return "\n";} else return "\n";}

function call_sikuli($input_intent,$input_params,$other_actions = '') { // helper function to use sikuli visual automation
if (strlen($input_intent)>9 and strtolower(substr($input_intent,-9))=='using ocr')
$use_ocr = "true"; else $use_ocr = "false"; // to track if it is a text locator using OCR
if (!touch('tagui.sikuli/tagui_sikuli.in')) die("ERROR - cannot initialise tagui_sikuli.in\n");
if (!touch('tagui.sikuli/tagui_sikuli.out')) die("ERROR - cannot initialise tagui_sikuli.out\n");
if ($other_actions != '') $other_actions = "\n" . $other_actions;
return "{techo('".str_replace(' to snap_image()','',$input_intent)."'); var fs = require('fs');\n" .
"if (!sikuli_step('".$input_intent."')) if (!fs.exists('".$input_params."') && !".$use_ocr.")\n" .
"{this.echo('ERROR - cannot find image file ".$input_params."').exit(); this.wait(0);} else\n" .
"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .

"error_webhook('123')".

"this.echo('ERROR - cannot find ".$input_params." on screen').exit(); this.wait(0);}" . $other_actions. "}" .
end_fi()."});\n\n";}

[kensoh]

CC @ruthtxh fyi

[marcelocecin]

hi Ken ! can I use the api_config this way ?

function error_webhook(error) {
    casper.echo('[WEBHOOK POST MESSAGE]\n' + error + '\n');
    var api_config = { method: "POST", header: [], body: {"error:"+error} };
    call_api('https://webhook.site/');
    casper.echo('[REST API RESPONSE]' + api_result + '\n');
    try {api_json = JSON.parse(api_result);} catch(e) {api_json = JSON.parse('null');}
    casper.echo(api_result);
}

[kensoh]

Hi Marcelo, oops I'm sorry I missed out putting the documentation link on API step here (I totally forgot about it after typing the long message 😅) - https://github.com/kelaberetiv/TagUI/tree/pre_v6#api

Below is the syntax -

api_config = {method:'PUT', header:['Header1: value1','Header2: value2'], body:{'id':123,'pwd':'abc'}};

Your line should be something like this, the error part shouldn't need a + and the : should be outside the quotes. var also shouldn't be used to define local variable because the function call_api() will need to access the api_config variable.

api_config = {method: "POST", header: [], body: {"error": error}};

Let us know how it goes!

[marcelocecin]

hi Ken ! the following error happened

[kensoh]

Hi Marcelo, did you follow the example here,

https://github.com/kelaberetiv/TagUI/issues/939#issuecomment-778662012

By including a dummy api step?

https://google.com

// dummy api step to allow cross-domain traffic
api https://jsonplaceholder.typicode.com/users

click abcdefwxyz

By default, TagUI doesn't allow cross-domain traffic access for security reasons (eg while at 1 website, accessing data or sending data to another website), because it could mean some security lapse on the main website. When you use api step in your workflow, TagUI recognises that you intentionally want to send or receive traffic to 2nd website while still at 1st website.

[marcelocecin]

Ah yes ! sorry, I forgot that detail now everything worked out thank you very much Ken!

[kensoh]

(re-opening issue for now because the commit for auto-screenshot is not yet released)

[marcelocecin]

hello ken I would like to suggest a new version for tagui_parse.php in this case, there would be no javascript code execution because I need to POST the API in multipart form-data format follow the suggestion:

function build_data_files($boundary, $fields, $files){
    $data = '';
    $eol = "\r\n";
    $delimiter = '-------------' . $boundary;
    foreach ($fields as $name => $content) {
        $data .= "--" . $delimiter . $eol
            . 'Content-Disposition: form-data; name="' . $name . "\"".$eol.$eol
            . $content . $eol;
    }
    foreach ($files as $name => $content) {
        $data .= "--" . $delimiter . $eol
            . 'Content-Disposition: form-data; name="' . $name . '"; filename="' . $name . '"' . $eol
            //. 'Content-Type: image/png'.$eol
            . 'Content-Transfer-Encoding: binary'.$eol
            ;
        $data .= $eol;
        $data .= $content . $eol;
    }
    $data .= "--" . $delimiter . "--".$eol;
    return $data;
}

function send_api_error($error,$file){
//$fields = array("f1"=>"value1", "another_field2"=>"anothervalue");
$fields = array("error"=>$error);
//$filenames = array("/tmp/1.jpg", "/tmp/2.png");
$filenames = array($file);
$files = array();
foreach ($filenames as $f){
   $files[$f] = file_get_contents($f);
}
$url = "https://webhook.site/";
$curl = curl_init();
$url_data = http_build_query($data);
$boundary = uniqid();
$delimiter = '-------------' . $boundary;
$post_data = build_data_files($boundary, $fields, $files);
curl_setopt_array($curl, array(
  CURLOPT_URL => $url,
  CURLOPT_RETURNTRANSFER => 1,
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 30,
  CURLOPT_CUSTOMREQUEST => "POST",
  CURLOPT_POST => 1,
  CURLOPT_POSTFIELDS => $post_data,
  CURLOPT_HTTPHEADER => array(
    //"Authorization: Bearer $TOKEN",
    "Content-Type: multipart/form-data; boundary=" . $delimiter,
    "Content-Length: " . strlen($post_data)
  ),
));
$response = curl_exec($curl);
$info = curl_getinfo($curl);
var_dump($response);
$err = curl_error($curl);
var_dump($err);
curl_close($curl);
}

function end_tx($locator) { // helper function to return ending string for handling locators
if ($GLOBALS['inside_while_loop'] == 0)
return "},\nfunction timeout() {this.capture('".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');".
send_api_error('ERROR - cannot find '.$locator,abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME)).'_error.png').
"\nthis.echo('ERROR - cannot find ".$locator."').exit(); this.wait(0);});}"."});".end_fi()."\n\n";
else if ($GLOBALS['inside_code_block']==0)
{$GLOBALS['inside_while_loop'] = 0; // reset inside_while_loop if not inside block
$GLOBALS['for_loop_tracker'] = ""; // reset for_loop_tracker if not inside block
return "\n";} else return "\n";}

function call_sikuli($input_intent,$input_params,$other_actions = '') { // helper function to use sikuli visual automation
$path = abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png";
$type = pathinfo($path, PATHINFO_EXTENSION);
$data = file_get_contents($path);
$base64 = 'data:image/' . $type . ';base64,' . base64_encode($data);
if (strlen($input_intent)>9 and strtolower(substr($input_intent,-9))=='using ocr')
$use_ocr = "true"; else $use_ocr = "false"; // to track if it is a text locator using OCR
if (!touch('tagui.sikuli/tagui_sikuli.in')) die("ERROR - cannot initialise tagui_sikuli.in\n");
if (!touch('tagui.sikuli/tagui_sikuli.out')) die("ERROR - cannot initialise tagui_sikuli.out\n");
if ($other_actions != '') $other_actions = "\n" . $other_actions;
return "{techo('".str_replace(' to snap_image()','',$input_intent)."'); var fs = require('fs');\n" .
"if (!sikuli_step('".$input_intent."')) if (!fs.exists('".$input_params."') && !".$use_ocr.")\n" .
"{error_webhook('ERROR - cannot find image file ".$input_params."');\n".
"this.echo('ERROR - cannot find image file ".$input_params."').exit(); this.wait(0);} else\n" .
"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .
send_api_error('ERROR - cannot find '.$input_params.' on screen',abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME)).'_error.png').
"this.echo('ERROR - cannot find ".$input_params." on screen').exit(); this.wait(0);}" . $other_actions. "}" .
end_fi()."});\n\n";}

but the error is occurring: PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /tagui/src/tagui_parse.php PS: I already ran the command sudo apt-get install php-curl on Linux thanks

[kensoh]

Hi Marcelo, interesting functions and design you created!

Oh TagUI can't swap to using curl-based design for api step. This is because, as you have found, the curl does not come with PHP unless it was built and compiled as part of the PHP installed. For eg the PHP that comes with macOS has curl but the PHP that comes with my CentOS server does not. So having a curl-based approach will make installation of TagUI more complex due to the need for users to install a version of PHP that has curl or additional setup for curl to be usable in PHP.

I'm not familiar with using curl within PHP enough to know why you received the above error, but below are walkthroughs which might reveal some additional steps needed to make curl_init() available. Before you add the code in tagui_parse.php, try to run a simple .php file to test first to make sure the curl sequences are working ok.

https://www.digitalocean.com/community/questions/curl-is-not-installed-in-your-php-installation https://www.geeksforgeeks.org/how-to-install-php-curl-in-ubuntu/

Let me know if you run into issues!

PS - adding on, if your environment has Python, you can also consider using the py begin and py finish steps in TagUI to do the API call using Python requests package. Here's an example to send binary form data with that package. My rationale for suggesting that is because that path has been widely used, so issues for usage and implementation would already be solved and may save you time writing own engine - https://stackoverflow.com/questions/14365027/python-post-binary-data

[marcelocecin]

hi Ken I understand why the ideal is to use Python a doubt, is there a way to use the py begin and py finish code inside the tagui_parse.php ? because the intention is to send the POST inside the end_tx and call_sikuli functions or is it right to run the exec function in PHP to execute a file in Python ?

[kensoh]

Oh for doing webhooks, the actual code should be in tagui_global.js, in the example here -

https://github.com/kelaberetiv/TagUI/issues/939#issuecomment-778662012

In tagui_parse.php it is only generating the JS code to be run by calling the webhook function during actual execution.

I see, if you want to override error handling, then I think inside tagui_global.js you can include the send_api_error() function, and inside that function you can use py_step() to send the python commands you want directly to Python to run. But in your workflow script will need some dummy Python step so that TagUI will know to create Python process in background.

It'll probably be something like -

// dummy python step in workflow.tag to trigger loading Python process
py a = 1

// do some steps
// ...
// ...

// on errror, the modified tagui_parse.php will call send_api_error(), which has to be defined in tagui_global.js

And in send_api_error(), py_step() can be used to run the Python commands by providing the command string as input.

For transferring files, probably you can use Python to open up the error image to do the base64 conversion before sending.

Above are the overview steps, but I don't know which way is the easiest because I have not tried doing this before.

TagUI's api step, in above example here https://github.com/kelaberetiv/TagUI/issues/939#issuecomment-779524819 can take in body data, but I have not tried sending base64 string data to an API before to know if that works.

[marcelocecin]

hi Ken, I tried using tagui_global.js I actually tried in various ways with javascript but none of them worked. follow new suggestion:

function end_tx($locator) { // helper function to return ending string for handling locators
if ($GLOBALS['inside_while_loop'] == 0)
return "},\nfunction timeout() {this.capture('".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');".
exec("curl -v -F error='ERROR - cannot find ".$locator."' -F upload=@".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME)).'_error.png'." https://webhook.site").
"\nthis.echo('ERROR - cannot find ".$locator."').exit(); this.wait(0);});}"."});".end_fi()."\n\n";
else if ($GLOBALS['inside_code_block']==0)
{$GLOBALS['inside_while_loop'] = 0; // reset inside_while_loop if not inside block
$GLOBALS['for_loop_tracker'] = ""; // reset for_loop_tracker if not inside block
return "\n";} else return "\n";}

function call_sikuli($input_intent,$input_params,$other_actions = '') { // helper function to use sikuli visual automation
if (strlen($input_intent)>9 and strtolower(substr($input_intent,-9))=='using ocr')
$use_ocr = "true"; else $use_ocr = "false"; // to track if it is a text locator using OCR
if (!touch('tagui.sikuli/tagui_sikuli.in')) die("ERROR - cannot initialise tagui_sikuli.in\n");
if (!touch('tagui.sikuli/tagui_sikuli.out')) die("ERROR - cannot initialise tagui_sikuli.out\n");
if ($other_actions != '') $other_actions = "\n" . $other_actions;
return "{techo('".str_replace(' to snap_image()','',$input_intent)."'); var fs = require('fs');\n" .
"if (!sikuli_step('".$input_intent."')) if (!fs.exists('".$input_params."') && !".$use_ocr.")\n" .
"{this.echo('ERROR - cannot find image file ".$input_params."').exit(); this.wait(0);} else\n" .
"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .
exec("curl -v -F error='ERROR - cannot find ".$input_params." on screen' -F upload=@".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME)).'_error.png'." https://webhook.site").
"this.echo('ERROR - cannot find ".$input_params." on screen!!!').exit(); this.wait(0);}" . $other_actions. "}" .
end_fi()."});\n\n";}

now POST is working, however it is running as soon as the TAG script is started, long before the error actually occurs.

thanks !

[kensoh]

Oh Marcelo, in the function call_sikuli(), what it does is generate the correct JavaScript code to be run during execution. But your use of exec() is PHP code that runs already before JavaScript runs. If using curl, you can try doing below -

"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .
"send_api_error('curl ...');\n" .

PS - make sure the single and double quotes are escaped correctly in above line so that the string is formed correctly

In tagui_global.js, you can define the function to run commands this way

function send_api_error(command) {
casper.waitForExec(command, null, function(response) {run_result = ''; 
run_result = (response.data.stdout.trim() || response.data.stderr.trim()); run_result = run_result.replace(/\r\n/g,'\n');
run_json = response.data;}, function() {this.echo('ERROR - command to run exceeded '+(casper.options.waitTimeout/1000).toFixed(1)+'s timeout').exit();},casper.options.waitTimeout);}
}

[marcelocecin]

hi Ken it's still not working the request is not arriving on the webhook I tested it via shell command and then it works

tagui_global.js

function send_webhook_error(error,image) {
  casper.echo(error);
  casper.echo(image);
  casper.waitForExec('curl -F error="'+error+'" -F upload=@'+image+' https://webhook.site', null,
  //casper.waitForExec('curl',['-F error="'+error+'"','-F upload=@'+image,'https://webhook.site'],
  //casper.waitForExec('curl -F error="'+error+'" -F upload=@'+image+' https://webhook.site', null,
    function(response) {
      run_result = '';
      run_result = (response.data.stdout.trim() || response.data.stderr.trim());
      run_result = run_result.replace(/\r\n/g,'\n');
      run_json = response.data;
      this.echo(run_result);
      this.echo(run_json);
    }, function(timeout, response) {
        this.echo("Program finished by casper:" + JSON.stringify(response.data));
  });
}

tagui_parse.php

function end_tx($locator) { // helper function to return ending string for handling locators
if ($GLOBALS['inside_while_loop'] == 0)
return "},\nfunction timeout() {this.capture('".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');".
"send_webhook_error('ERROR - cannot find ".$input_params." on screen','".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n".
"\nthis.echo('ERROR - cannot find ".$locator."').exit(); this.wait(0);});}"."});".end_fi()."\n\n";
else if ($GLOBALS['inside_code_block']==0)
{$GLOBALS['inside_while_loop'] = 0; // reset inside_while_loop if not inside block
$GLOBALS['for_loop_tracker'] = ""; // reset for_loop_tracker if not inside block
return "\n";} else return "\n";}

function call_sikuli($input_intent,$input_params,$other_actions = '') { // helper function to use sikuli visual automation
if (strlen($input_intent)>9 and strtolower(substr($input_intent,-9))=='using ocr')
$use_ocr = "true"; else $use_ocr = "false"; // to track if it is a text locator using OCR
if (!touch('tagui.sikuli/tagui_sikuli.in')) die("ERROR - cannot initialise tagui_sikuli.in\n");
if (!touch('tagui.sikuli/tagui_sikuli.out')) die("ERROR - cannot initialise tagui_sikuli.out\n");
if ($other_actions != '') $other_actions = "\n" . $other_actions;
return "{techo('".str_replace(' to snap_image()','',$input_intent)."'); var fs = require('fs');\n" .
"if (!sikuli_step('".$input_intent."')) if (!fs.exists('".$input_params."') && !".$use_ocr.")\n" .
"{this.echo('ERROR - cannot find image file ".$input_params."').exit(); this.wait(0);} else\n" .
"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .
"send_webhook_error('ERROR - cannot find ".$input_params." on screen','".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n".
"this.echo('ERROR - cannot find ".$input_params." on screen!!!').exit(); this.wait(0);}" . $other_actions. "}" .
end_fi()."});\n\n";}

9_webhook.tag

https://google.com
timeout 5
click webhook.png
echo DONE

output on executing script:

funny that echo DONE doesn't appear at the end of the script execution

thanks

[kensoh]

Oh in below line, the exit() means quit once the error is encountered. So there is no chance of reaching echo DONE

"this.echo('ERROR - cannot find ".$input_params." on screen!!!').exit(); this.wait(0);}" . $other_actions. "}" .

Is there anything returned from run_result? Below is a workflow using run step and the generated JavaScript file that works. An abc.txt file is created in tagui/src folder after running. This means the curl command works. Maybe you can try if this works for your environment, and see what is the part that makes your custom function calling curl fail.

demo.tag

run curl -k -o abc.txt https://raw.githubusercontent.com/kelaberetiv/TagUI/master/src/erina
echo `run_result`

demo.js

casper.then(function() {techo('run curl -k -o abc.txt https://raw.githubusercontent.com/kelaberetiv/TagUI/master/src/erina');});
casper.then(function() {casper.waitForExec('curl -k -o abc.txt https://raw.githubusercontent.com/kelaberetiv/TagUI/master/src/erina', null, function(response) {run_result = '';
run_result = (response.data.stdout.trim() || response.data.stderr.trim()); run_result = run_result.replace(/\r\n/g,'\n');
run_json = response.data;}, function() {this.echo('ERROR - command to run exceeded '+(casper.options.waitTimeout/1000).toFixed(1)+'s timeout').exit();},casper.options.waitTimeout);});

[kensoh]

Hope with above, you'll find some answer Marcelo! From looking at your code I can't spot any problems.

[marcelocecin]

hi Ken your suggestion helped in part, I ended up discovering that to avoid error with the error script, the ideal is to encode it in base64 but the main error still remains, it appears that the browser is closing before the curl request is executed

tagui_parse.php

function call_sikuli($input_intent,$input_params,$other_actions = '') { // helper function to use sikuli visual automation
if (strlen($input_intent)>9 and strtolower(substr($input_intent,-9))=='using ocr')
$use_ocr = "true"; else $use_ocr = "false"; // to track if it is a text locator using OCR
if (!touch('tagui.sikuli/tagui_sikuli.in')) die("ERROR - cannot initialise tagui_sikuli.in\n");
if (!touch('tagui.sikuli/tagui_sikuli.out')) die("ERROR - cannot initialise tagui_sikuli.out\n");
if ($other_actions != '') $other_actions = "\n" . $other_actions;
return "{techo('".str_replace(' to snap_image()','',$input_intent)."'); var fs = require('fs');\n" .
"if (!sikuli_step('".$input_intent."')) if (!fs.exists('".$input_params."') && !".$use_ocr.")\n" .
"{this.echo('ERROR - cannot find image file ".$input_params."').exit(); this.wait(0);} else\n" .
"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .
"send_webhook_error('ERROR','".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n".
"this.echo('ERROR - cannot find ".$input_params." on screen!!!').exit(); this.wait(0);}" . $other_actions. "}" .
end_fi()."});\n\n";}

tagui_global.js

function send_webhook_error(error,image) {
  casper.echo(error);
  casper.echo(image);
  casper.waitForExec('curl -F error="Error" -F upload=@'+image+' https://webhook.site/53388dfe-1308-45ae-8c88-c31b89c10cbb', null, function(response) {
      this.echo('running CURL');
      run_result = '';
      run_result = (response.data.stdout.trim() || response.data.stderr.trim()); run_result = run_result.replace(/\r\n/g,'\n');
      run_json = response.data;},
      function() {this.echo('ERROR - command to run exceeded '+(casper.options.waitTimeout/1000).toFixed(1)+'s timeout').exit();});
}

note that it did not enter the function, as it did not show running CURL

I created a tag file to test

demo.tag

run curl -F error="ERROR" -F upload=@/home/cdj/Downloads/tagui/flows/samples/9_webhook_error.png https://webhook.site/53388dfe-1308-45ae-8c88-c31b89c10cbb
echo `run_result`

and it worked

I really don't know what could be happening thanks !

[marcelocecin]

hi Ken here is the suggestion, now is working in both scenarios: visual and headless

tagui_global.js

function send_webhook_error(error,image) {
casper.waitForExec('curl -F error="'+error+'" -F upload=@'+image+' https://webhook.site/', null, function(response) {
run_result = '';
run_result = (response.data.stdout.trim() || response.data.stderr.trim()); run_result = run_result.replace(/\r\n/g,'\n');
run_json = response.data;
this.exit();},
function() {this.echo('ERROR - command to run exceeded '+(casper.options.waitTimeout/1000).toFixed(1)+'s timeout').exit();});
}

tagui_parse.php

function end_tx($locator) { // helper function to return ending string for handling locators
$wh_error = base64_encode('ERROR - cannot find '.$locator);
if ($GLOBALS['inside_while_loop'] == 0)
return "},\nfunction timeout() {this.capture('".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');".
"send_webhook_error('".$wh_error."','".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n".
"\nthis.echo('ERROR - cannot find ".$locator."'); this.wait(0);});}"."});".end_fi()."\n\n";
else if ($GLOBALS['inside_code_block']==0)
{$GLOBALS['inside_while_loop'] = 0; // reset inside_while_loop if not inside block
$GLOBALS['for_loop_tracker'] = ""; // reset for_loop_tracker if not inside block
return "\n";} else return "\n";}

function call_sikuli($input_intent,$input_params,$other_actions = '') { // helper function to use sikuli visual automation
$wh_error = base64_encode('ERROR - cannot find '.$input_params.' on screen');
if (strlen($input_intent)>9 and strtolower(substr($input_intent,-9))=='using ocr')
$use_ocr = "true"; else $use_ocr = "false"; // to track if it is a text locator using OCR
if (!touch('tagui.sikuli/tagui_sikuli.in')) die("ERROR - cannot initialise tagui_sikuli.in\n");
if (!touch('tagui.sikuli/tagui_sikuli.out')) die("ERROR - cannot initialise tagui_sikuli.out\n");
if ($other_actions != '') $other_actions = "\n" . $other_actions;
return "{techo('".str_replace(' to snap_image()','',$input_intent)."'); var fs = require('fs');\n" .
"if (!sikuli_step('".$input_intent."')) if (!fs.exists('".$input_params."') && !".$use_ocr.")\n" .
"{this.echo('ERROR - cannot find image file ".$input_params."').exit(); this.wait(0);} else\n" .
"{sikuli_step('snap page.png to ".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n" .
"send_webhook_error('".$wh_error."','".abs_file(pathinfo($GLOBALS['script'],PATHINFO_FILENAME))."_error.png');\n".
"this.echo('ERROR - cannot find ".$input_params." on screen'); this.wait(0);}" . $other_actions. "}" .
end_fi()."});\n\n";}

the first thing is to encode the error string to base64, stored in the variable $wh_error

the second thing was to remove the .exit() function from the end of the this.echo() function call; as this was preventing the send_webhook_error function from running

and last but not least, include in the function send_webhook_error the command this.exit(); to finish code execution

thanks !

[kensoh]

Hi Marcelo, thanks for sharing!! I see.. I understand now, sounds like probably the way the JavaScript code is ran, the exit() got called even before the send_webhook_error() function.

I'll save your solution here for now, but do recommend that you merge it into your fork of TagUI. For now, can't merge back to master because the implementation endpoints will be different for different users and most users would probably not want to send data to an external API service on error.

[marcelocecin]

hi Ken I completely understand later, when I have more time, I'll think of a way to declare two variables: one to detect if the user wants this function to be active and if so what would be the webhook's url now I'm going to start using RPA-Python and I'm creating an API service via Flask to run the tasks on demand when I'm done I'll share an example in the other repository. thanks !

[kensoh]

Thank Marcelo for trying out the Python version!

Oh for TagUI human language version, you can modify to merge to your fork. For now it will be hard to merge to TagUI main branch because even if there is option to choose webhook URL, the parameters to send are quite varied. So this will be a feature more for developer users to use, and not so straightforward to configure through a TagUI step.