aisingapore / TagUI

Free RPA tool by AI Singapore
Apache License 2.0
5.65k stars 585 forks source link

Recording of file input selection cannot work #48

Closed prinspark closed 7 years ago

prinspark commented 7 years ago

I'm back,kensoh:) I wanna upload an image to server by file input box recently , but TAGUI just record 'click .file-upload-btn input'. Is it a defect of casperjs?

kensoh commented 7 years ago

Hi @prinspark it has been a while :)

I see, uploading files is usually restricted by automation due to security reasons. Imagine a JS script has the ability to upload some private file on your laptop to their web-server. So by design, most browsers force this to be done by user manual action. For the TagUI Chrome extension, it is based on Resurrectio project and also does not record the OS events when user choose a file from the file manager, because during headless automation this file manager popup does not happen.

There is some method provided by PhantomJS and Chrome also has made a provision for setting this manually without simulating that file explorer popup box. I recently added a new TagUI step upload to do that. You can try if that works for you. Syntax is upload css-selector as filename

https://github.com/tebelorg/TagUI#steps-description

prinspark commented 7 years ago

upload .file-upload-btn ~/imagetopost.jpg Is that right?

prinspark commented 7 years ago

Or use full css selector below?

app > div > from > div:nth-child(10) > div > div > div > div:nth-child(1) > div.file-upload-btn.has-img > input[type="file"]

kensoh commented 7 years ago

Not sure if ~ will work for the file name expansion, you can try upload .file-upload-btn input as filename where .file-upload-btn input is the css-selector for the file input button, and filename can be relative path or full path on your laptop.

During testing, I validated the step works for imgur.com for both PhantomJS and Chrome.

kensoh commented 7 years ago

If file-upload-btn is the classname .file-upload-btn might be able to work already.

When I tried with imgur.com don't need full path of css. If it is ID identifier it will be #element

prinspark commented 7 years ago

ok , and how to restrict click href action to open within current window or page?

kensoh commented 7 years ago

I think that's controlled purely by the website. There is attribute to force open on a new window/tab versus on the same window/tab. If you are familiar with CasperJS and running TagUI in default mode (CasperJS + PhantomJS) then you can try hacking using below function. May be try getting that href value into a variable. http://docs.casperjs.org/en/latest/modules/casper.html#getelementattribute

Then decompose to get the domain name + rest of URL, or form an URL if the href is relative href.

Then use the navigation step https:// or http://'+variable+' to go to that URL directly. This is quite hacky and not straightforward but may be possible if you are familiar with CasperJS and if that webpage able to retrieve that href value.

prinspark commented 7 years ago

The recorded flow file stuck in element locating when jump to a new page.So I guess, it might be caused by this reason. Any solution?

kensoh commented 7 years ago

Do you mean that recording fails once you clicked on a link to a new page, and the things you do on the new page cannot be recorded? If yes, that's expected as the Resurrectio recorder cannot record new events on the new popups, I believe. It might be able to in some cases, but I don't think it is supported in the original design.

The steps after that popup window might still be able to enter manually, but you will need to use popup step to specify some URL string to look for in the popup window, so that the next step will be executed in that popup window. If there are multiple steps you can do like below -

popup string_in_url_of_popup_window
{
click aaa
type bbb as ccc
click ddd
}

https://github.com/tebelorg/TagUI#steps-description

Also, the recording is not foolproof, more to help reduce typing manually all the steps and identifier. Normally for automation, I still prefer to do it manually using the web browser inspector to select the best identifier to use.

Base on the scenario I can decide whether to make it very easy to fail by being very specific (eg during testing I want to make sure if a small change happens I want to detect failure), but for automation maybe make it less likely to fail (eg specifying username as the selector may still work is there is some small change of identifier to username_field).

prinspark commented 7 years ago

Nope, it can be recorded. But when I try to replay the flow file, it responses an error 'can not find element'.

kensoh commented 7 years ago

Did you try adding the popup step like above comment?

kensoh commented 7 years ago

Yeah just checked the Resurrectio source code, can't work properly with popups or framesets

//caveats: popup windows undefined, cant handle framesets

prinspark commented 7 years ago

So it means automation can't go on unless all actions stay in one tag page?

kensoh commented 7 years ago

Did you try adding the popup step like above comment? Need to use popup step to define the context which tab / webpage you want to automate. Otherwise CasperJS or Chrome won't know which element you are referring to on the webpage if there are multiple web pages open.

popup string_in_url_of_popup_window
{
click aaa
type bbb as ccc
click ddd
}
kensoh commented 7 years ago

Closing the issue for now, as recording popup is not supported by Resurrectio (the tool that the Chrome extension is base on). Also in readme, there's a note that recording is not foolproof, meant to reduce manual typing. More to record outline then edit for adjustment.

For manual editing, popup step and frame step can be used to set the context of where to take action. A lot of repeated details already above, and in readme steps description section. Tried my best to provide details and tips on what to try 😞 But if after so much exchange the idea cannot get through then another 20 messages will not make any difference.