search

tebelorg / RPA-Python

Python package for doing RPA
Apache License 2.0
4.84k stars 664 forks source link

frame() failed for iframe in different domain - pending replication #476

Closed taoheihei closed 1 year ago

taoheihei commented 1 year ago

hello , I have tested two pages ,the code examples are as below, is there any way to operate the frame document? page a

<html>
<body>
    <iframe width="100%"  height="100%"  src="http://page_b.google.com"></iframe>
</body>
</html>

page b

<html>
<body>
    <div id="app">
           <input type="text" id="in" />
      </div>
</body>
</html>

and the python code

import rpa as r
r.init()
r.url('http://page_a.google.com')
r.dom('document.getElementsByTagName("iframe")[0].id = "test"')
#r.dom('document.domain="google.com"')
switch = r.frame('test')
r.wait(1)
print(switch)
r.type('//*[@id="app"]//input', 'test')
print(r.present('//*[@id="app"]'))

switch is True, but error in next step, it may be failed by the iframe Same-origin policy,

kensoh commented 1 year ago

Hi @taoheihei thanks for raising this and sharing above samples! Do you have a website that can be tested? Above won't be able to test locally. I have not designed the tool to support cross-site iframe access, so not sure if that will work at all.

If the bottleneck is Chrome web automation policy there is little that can be done. If not, there could be possible workarounds. If you don't have a public site example that can be used for testing above, could you run with r.debug(True) and share what you see? Also, if you are running on Windows can you share the error message from the Chrome Engine window? For Mac, the log is in print(r.tagui_location()) folder --> .tagui/src/tagui_chrome.log (for Windows it is in %appdata%/tagui/src/tagui_chrome.log

kensoh commented 1 year ago

In the meantime, possible workarounds could be:

taoheihei commented 1 year ago

@kensoh sorry for seeing your message late, I have debuged , and this is the debug log and chrome_log

[RPA][1] - listening for inputs
[RPA][2] - dump url() to rpa_python.txt
[RPA][2] - listening for inputs
[RPA][3] - exist_result = exist('userName').toString()
[RPA][3] - listening for inputs
[RPA][4] - dump exist_result to rpa_python.txt
[RPA][4] - listening for inputs
[RPA][5] - type userName as fulinglin
[RPA][5] - listening for inputs
[RPA][6] - exist_result = exist('password').toString()
[RPA][6] - listening for inputs
[RPA][7] - dump exist_result to rpa_python.txt
[RPA][7] - listening for inputs
[RPA][8] - type password as fll2017400181fll
[RPA][8] - listening for inputs
[RPA][9] - exist_result = exist('//button[@type="submit"]').toString()
[RPA][9] - listening for inputs
[RPA][10] - dump exist_result to rpa_python.txt
[RPA][10] - listening for inputs
[RPA][11] - click //button[@type="submit"]
[RPA][11] - listening for inputs
[RPA][12] - http://sjcm.jianke-inc.com/BIReport
[RPA][12] - listening for inputs
[RPA][ERROR] - 'utf-8' codec can't decode bytes in position 65-66: invalid continuation byte
[RPA][13] - listening for inputs
[RPA][14] - dump present_result to rpa_python.txt
[RPA][14] - listening for inputs
[RPA][ERROR] - 'utf-8' codec can't decode bytes in position 61-62: invalid continuation byte
[RPA][15] - listening for inputs
[RPA][16] - dump exist_result to rpa_python.txt
[RPA][16] - listening for inputs
[RPA][ERROR] - 'utf-8' codec can't decode bytes in position 45-46: invalid continuation byte
[RPA][17] - listening for inputs
[RPA][18] - present_result = present('//span[@title="用户数据" and contains(@class, "ant-tree-node-content-wrapper-close")]').toString()
[RPA][18] - listening for inputs
[RPA][19] - dump present_result to rpa_python.txt
[RPA][19] - listening for inputs
[RPA][20] - exist_result = exist('//span[@title="用户数据" and contains(@class, "ant-tree-node-content-wrapper-close")]').toString()
[RPA][20] - listening for inputs
[RPA][21] - dump exist_result to rpa_python.txt
[RPA][21] - listening for inputs
[RPA][22] - click //span[@title="用户数据" and contains(@class, "ant-tree-node-content-wrapper-close")]
[RPA][22] - listening for inputs
[RPA][23] - present_result = present('//span[@title="搜索数据" and contains(@class, "ant-tree-node-content-wrapper-close")]').toString()
[RPA][23] - listening for inputs
[RPA][24] - dump present_result to rpa_python.txt
[RPA][24] - listening for inputs
[RPA][25] - exist_result = exist('//span[@title="搜索数据" and contains(@class, "ant-tree-node-content-wrapper-close")]').toString()
[RPA][25] - listening for inputs
[RPA][26] - dump exist_result to rpa_python.txt
[RPA][26] - listening for inputs
[RPA][27] - click //span[@title="搜索数据" and contains(@class, "ant-tree-node-content-wrapper-close")]
[RPA][27] - listening for inputs
[RPA][ERROR] - 'utf-8' codec can't decode bytes in position 67-68: invalid continuation byte
[RPA][28] - listening for inputs
[RPA][29] - dump exist_result to rpa_python.txt
[RPA][29] - listening for inputs
[RPA][ERROR] - 'utf-8' codec can't decode bytes in position 51-52: invalid continuation byte
[RPA][30] - listening for inputs
[RPA][31] - dom document.getElementsByTagName("iframe")[0].id = "keyword"
[RPA][31] - listening for inputs
[RPA][32] - dump dom_result to rpa_python.txt
[RPA][32] - listening for inputs
[RPA][33] - dom document.domain ="jianke-inc.com"
[RPA][33] - listening for inputs
[RPA][34] - dump dom_result to rpa_python.txt
[RPA][34] - listening for inputs
[RPA][35] - js chrome_step("Runtime.evaluate", {expression: "mainframe_context = null"})
[RPA][35] - listening for inputs
[RPA][36] - js chrome_step("Runtime.evaluate", {expression: "subframe_context = null"})
[RPA][36] - listening for inputs
[RPA][37] - js chrome_context = "document"; frame_step_offset_x = 0; frame_step_offset_y = 0;
[RPA][37] - listening for inputs
[RPA][38] - exist_result = exist('(//frame|//iframe)[@name="keyword" or @id="keyword"]').toString()
[RPA][38] - listening for inputs
[RPA][39] - dump exist_result to rpa_python.txt
[RPA][39] - listening for inputs
[RPA][40] - js new_context = "mainframe_context"
[RPA][40] - listening for inputs
[RPA][41] - js frame_xpath = '(//frame|//iframe)[@name="keyword" or @id="keyword"]'
[RPA][41] - listening for inputs
[RPA][42] - js frame_rect = chrome.getRect(xps666(frame_xpath))
[RPA][42] - listening for inputs
[RPA][43] - js frame_step_offset_x = frame_rect.left; frame_step_offset_y = frame_rect.top;
[RPA][43] - listening for inputs
[RPA][44] - js chrome_step("Runtime.evaluate", {expression: new_context + " = document.evaluate('" + frame_xpath + "'," + chrome_context + ",null,XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,null).snapshotItem(0).contentDocument"})
[RPA][44] - listening for inputs
[RPA][45] - js chrome_context = new_context
[RPA][45] - listening for inputs
[RPA][ERROR] - 'utf-8' codec can't decode bytes in position 73-74: invalid continuation byte
[RPA][46] - listening for inputs
[RPA][47] - dump exist_result to rpa_python.txt
[RPA][47] - listening for inputs
[RPA][48] - js chrome_step("Runtime.evaluate", {expression: "mainframe_context = null"})
[RPA][48] - listening for inputs
[RPA][49] - js chrome_step("Runtime.evaluate", {expression: "subframe_context = null"})
[RPA][49] - listening for inputs
[RPA][50] - js chrome_context = "document"; frame_step_offset_x = 0; frame_step_offset_y = 0;
[RPA][50] - listening for inputs
[tagui] INPUT  - 
[135] {"id":135,"method":"Runtime.evaluate","params":{"expression":"document.evaluate('//span[@value=\"关键词搜索结果统计\"]',mainframe_context,null,XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,null).snapshotLength"}}
[tagui] OUTPUT - 
[135] {"id":135,"result":{"result":{"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Failed to execute 'evaluate' on 'Document': parameter 2 is not of type 'Node'.\n    at <anonymous>:1:10","objectId":"2357429549606851384.9.89"},"exceptionDetails":{"exceptionId":50,"text":"Uncaught","lineNumber":0,"columnNumber":9,"scriptId":"79","exception":{"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Failed to execute 'evaluate' on 'Document': parameter 2 is not of type 'Node'.\n    at <anonymous>:1:10","objectId":"2357429549606851384.9.90"}}}}

[tagui] INPUT  - 
[136] {"id":136,"method":"Runtime.evaluate","params":{"expression":"mainframe_context = null"}}
[tagui] OUTPUT - 
[136] {"id":136,"result":{"result":{"type":"object","subtype":"null","value":null}}}

[tagui] INPUT  - 
[137] {"id":137,"method":"Runtime.evaluate","params":{"expression":"subframe_context = null"}}
[tagui] OUTPUT - 
[137] {"id":137,"result":{"result":{"type":"object","subtype":"null","value":null}}}

Hi @taoheihei thanks for raising this and sharing above samples! Do you have a website that can be tested? Above won't be able to test locally. I have not designed the tool to support cross-site iframe access, so not sure if that will work at all.

If the bottleneck is Chrome web automation policy there is little that can be done. If not, there could be possible workarounds. If you don't have a public site example that can be used for testing above, could you run with r.debug(True) and share what you see? Also, if you are running on Windows can you share the error message from the Chrome Engine window? For Mac, the log is in print(r.tagui_location()) folder --> .tagui/src/tagui_chrome.log (for Windows it is in %appdata%/tagui/src/tagui_chrome.log

kensoh commented 1 year ago

Hi @taoheihei above looks like the cross-site frame is not accessible. Do you have a public website and replication code that I can try to run to confirm this? Also, from your log your PC seems to have encoding incompatibility, see this link below to fix:

https://github.com/tebelorg/RPA-Python/issues/451#issuecomment-1556169481

kensoh commented 1 year ago

(If you are using non-English operating system and get "invalid continuation byte" error, you can set code page to support UTF-8 or change your Python script's encoding to your OS encoding. https://github.com/tebelorg/RPA-Python/issues/451#issuecomment-1556169481.)

kensoh commented 1 year ago

Closing issue for now, but do share back here!