Closed alimayorru closed 1 year ago
please provide a traceback
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\driverless.py", line 39, in <module>
elements = driver.find_elements(By.TAG_NAME, "h3") # Finds all elements in the page
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\sync\webdriver.py", line 37, in syncified
return self._loop.run_until_complete(item(*args, **kwargs))
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete
return future.result()
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\webdriver.py", line 780, in find_elements
return await parent.find_elements(by=by, value=value)
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\webelement.py", line 155, in find_elements
return await self.execute_script("return this.getElementsByTagName(arguments[0])",
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\__init__.py", line 65, in execute_script
obj_id = await self.obj_id
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\webelement.py", line 103, in obj_id
res = await self._driver.execute_cdp_cmd("DOM.resolveNode", {"nodeId": self._node_id})
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\webdriver.py", line 1295, in execute_cdp_cmd
result = await socket.exec(method=cmd, params=cmd_args, timeout=self._script_timeout)
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\cdp_socket\socket.py", line 69, in exec
return await asyncio.wait_for(self._responses[_id], timeout=timeout)
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\tasks.py", line 445, in wait_for
return fut.result()
cdp_socket.exceptions.CDPError: {'code': -32000, 'message': 'No node with given id found'}
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001E3C9520B80>
Traceback (most recent call last):
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x000001E3D9A318C0>
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1338, in close_connection
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1356, in close_transport
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1380, in wait_for_connection_lost
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\async_timeout.py", line 74, in timeout
RuntimeError: no running event loop
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001E3C9520B80>
Traceback (most recent call last):
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x000001E3D9A31540>
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1338, in close_connection
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1356, in close_transport
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1380, in wait_for_connection_lost
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\async_timeout.py", line 74, in timeout
RuntimeError: no running event loop
hmm I think I see the issue. I'll have a look at it in some time.
Yep, I can indeed reproduce this
@alimayorru This issue occured because events don't seem to be working in the sync version. I managed to get the following script (with patches which will be in the next release) working tho:
import time
from selenium_driverless.sync.webdriver import Chrome
from selenium_driverless.types.by import By
driver = Chrome()
driver.get("https://www.google.com")
driver.implicitly_wait(5)
reject_cookies = driver.find_element(By.XPATH,
"/html/body/div[2]/div[3]/div[3]/span/div/div/div/div[3]/div[1]/button[1]/div")
reject_cookies.click()
driver.implicitly_wait(1)
search_box = driver.find_element("name", "q")
search_box.write("test\n")
# Find the search button
search_button = driver.find_element(By.XPATH, "/html/body/div[1]/div[3]/form/div[1]/div[1]/div[4]/center/input[1]")
search_button.click()
driver.implicitly_wait(7)
elements = driver.find_elements(By.TAG_NAME, "a") # Finds all elements in the page
for t in elements:
jj = t.get_attribute('href')
if jj and "test" in jj:
# do something
Note: Should get resolved in the next selenium_driverless
release
@alimayorru feel free to report in case this doesn't resolve your issue
@alimayorru This issue occured because events don't seem to be working in the sync version. I managed to get the following script (with patches which will be in the next release) working tho:
import time from selenium_driverless.sync.webdriver import Chrome from selenium_driverless.types.by import By driver = Chrome() driver.get("https://www.google.com") driver.implicitly_wait(5) reject_cookies = driver.find_element(By.XPATH, "/html/body/div[2]/div[3]/div[3]/span/div/div/div/div[3]/div[1]/button[1]/div") reject_cookies.click() driver.implicitly_wait(1) search_box = driver.find_element("name", "q") search_box.write("test\n") # Find the search button search_button = driver.find_element(By.XPATH, "/html/body/div[1]/div[3]/form/div[1]/div[1]/div[4]/center/input[1]") search_button.click() driver.implicitly_wait(7) elements = driver.find_elements(By.TAG_NAME, "a") # Finds all elements in the page for t in elements: jj = t.get_attribute('href') if jj and "test" in jj: # do something
Note: Should get resolved in the next
selenium_driverless
release
your example is not making use of Selenium profile, my code i'm using selenium profile
@alimayorru
your example is not making use of Selenium profile, my code i'm using selenium profile
Yeah, I am aware of that. However, selenium-profiles
with driverless_options=True
uses internally the synchronous version of selenium-driverless
.
Therefore, if you install pip install selenium-driverless==1.4
, your issue should be resolved.
Feel free to report if that isn't the case.
with the same code i provided, i get this error after installing selenium-driverless==1.4.
C:\Users\alima\PycharmProjects\pythonProject\venv\Scripts\python.exe C:\Users\alima\PycharmProjects\pythonProject\driverless.py
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\driverless.py", line 21, in <module>
search_box = driver.find_element("name", "q")
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\sync\webdriver.py", line 51, in syncified
return self._loop.run_until_complete(item(*args, **kwargs))
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete
return future.result()
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\webdriver.py", line 794, in find_element
return await parent.find_element(by=by, value=value)
AttributeError: 'int' object has no attribute 'find_element'
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000015CCCF40C10>
Traceback (most recent call last):
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x0000015CECB9A0A0>
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1338, in close_connection
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1356, in close_transport
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1380, in wait_for_connection_lost
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\async_timeout.py", line 74, in timeout
RuntimeError: no running event loop
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x0000015CCCF40C10>
Traceback (most recent call last):
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x0000015CECB99D20>
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1338, in close_connection
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1356, in close_transport
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1380, in wait_for_connection_lost
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\async_timeout.py", line 74, in timeout
RuntimeError: no running event loop
Process finished with exit code 1
@alimayorru Sorry my bad, that was a typo:)
Tested it myself with selenium-driverless==1.4.1
, and it seems to be working
@alimayorru Sorry my bad, that was a typo:)
Tested it myself with
selenium-driverless==1.4.1
, and it seems to be working
Where's the version 1.4.1?. The latest on the store is 1.4
@alimayorru Sorry my bad, that was a typo:) Tested it myself with
selenium-driverless==1.4.1
, and it seems to be workingWhere's the version 1.4.1?. The latest on the store is 1.4
@alimayorru
1.4.1
correctly1.4
, but maybe it's just simplified or hasn't synced yet:)Is your issue resloved then? :)
I don't know why i'm having errors, new error is
C:\Users\alima\PycharmProjects\pythonProject\venv\Scripts\python.exe C:\Users\alima\PycharmProjects\pythonProject\driverless.py
C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\webelement.py:180: UserWarning: By.XPATH might be detectable, you might use driver.search_elements("//*[@name="q"]") or By.CSS_SELECTOR instead
warnings.warn(
C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\webelement.py:180: UserWarning: By.XPATH might be detectable, you might use driver.search_elements("/html/body/div[1]/div[3]/form/div[1]/div[1]/div[4]/center/input[1]") or By.CSS_SELECTOR instead
warnings.warn(
C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\webelement.py:153: UserWarning: By.TAG_NAME might be detectable, you might use driver.search_elements("h3") or By.CSS_SELECTOR instead
warnings.warn(
C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\selenium_driverless\types\webelement.py:209: UserWarning: This executes a script and can make you detected. You might use elem.get_dom_attribute instead if the attribute belongs to DOM.
You can pass warn=False to supress that warning.
warnings.warn("This executes a script and can make you detected. You might use elem.get_dom_attribute instead if the attribute belongs to DOM. \n You can pass warn=False to supress that warning.")
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\driverless.py", line 36, in <module>
if "test" in jj:
TypeError: argument of type 'NoneType' is not iterable
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001F7D6E00C10>
Traceback (most recent call last):
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x000001F7F6A5E1F0>
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1338, in close_connection
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1356, in close_transport
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1380, in wait_for_connection_lost
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\async_timeout.py", line 74, in timeout
RuntimeError: no running event loop
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001F7D6E00C10>
Traceback (most recent call last):
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 750, in call_soon
File "C:\Users\alima\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 515, in _check_closed
RuntimeError: Event loop is closed
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x000001F7F6A5DE70>
Traceback (most recent call last):
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1338, in close_connection
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1356, in close_transport
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\protocol.py", line 1380, in wait_for_connection_lost
File "C:\Users\alima\PycharmProjects\pythonProject\venv\lib\site-packages\websockets\legacy\async_timeout.py", line 74, in timeout
RuntimeError: no running event loop
Process finished with exit code 1
@alimayorru The following lines:
if "test" in jj:
TypeError: argument of type 'NoneType' is not iterable
indicate that jj = t.get_attribute('href')
returned None
, as there's no href
for that element:)
This time I think it's not an issue in my package:)
Okay yea i think it's from my code.
how do i click the Enter key in this package because "search_box.send_keys(Keys.ENTER) " is not working
search_box.submit()
worked for me
I wouldn't use Keys.ENTER
or search_box.submit()
.
I'd reccommed elem.write("\n")
or just clicking on the search button for now
I'd reccommed elem.write("\n") or just clicking on the search button for now
elem.write("\n")
doesn't search it, instead opens a new line under the texts
I'd reccommed elem.write("\n") or just clicking on the search button for now
elem.write("\n")
doesn't search it, instead opens a new line under the texts
Oh that's interesting:) well then use whatever works for you:) I'll implement dispatching key events on Elements, maybe that's gonna resolve this then
I'd reccommed elem.write("\n") or just clicking on the search button for now
elem.write("\n")
doesn't search it, instead opens a new line under the textsOh that's interesting:) well then use whatever works for you:) I'll implement dispatching key events on Elements, maybe that's gonna resolve this then
ok
hello @kaliiiiiiiiii , still on the issues.
i'm trying to iterate through a list of iframe elements and click the ones that are clickable.
code works well with normal selenium, but it has never worked with selenium-driverless (always opting different errors).
i will provide both codes so you can try to reproduce the errors and know what's causing it.
NORMAL SELENIUM CODE
from selenium.webdriver.common.by import By
import time, random
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.ablison.com/pros-and-cons-of-being-a-court-reporter/")
time.sleep(15)
iframes = driver.find_elements(By.TAG_NAME, "iframe")
print(len(iframes))
for iframe in iframes:
try:
i = random.choice(iframes)
iframes.remove(i)
print(len(iframes))
i.click()
print("CLICKABLE")
break
except Exception as e:
print(e)
continue
time.sleep(15)
driver.quit()
SELENIUM-DRIVERLESS CODE
import time, random
from selenium_profiles.webdriver import Chrome
from selenium_profiles.profiles import profiles
from selenium_driverless.webdriver import ChromeOptions
from selenium.webdriver.common.by import By
profile = profiles.Windows() # or .Android
options = ChromeOptions()
driver = Chrome(profile, options=options, driverless_options=True)
driver.get("https://www.ablison.com/pros-and-cons-of-being-a-court-reporter/")
time.sleep(15)
iframes = driver.find_elements(By.TAG_NAME, "iframe")
print(len(iframes))
for iframe in iframes:
try:
i = random.choice(iframes)
iframes.remove(i)
print(len(iframes))
i.click()
print("CLICKABLE")
break
except Exception as e:
print(e)
continue
time.sleep(15)
driver.quit()
hello @kaliiiiiiiiii , still on the issues.
i'm trying to iterate through a list of iframe elements and click the ones that are clickable.
code works well with normal selenium, but it has never worked with selenium-driverless (always opting different errors).
i will provide both codes so you can try to reproduce the errors and know what's causing it.
@alimayorru Sure, I can have a look at it:)
Can you provide me the errors (including tracebacks) for the driverless script as well?
hello @kaliiiiiiiiii , still on the issues. i'm trying to iterate through a list of iframe elements and click the ones that are clickable. code works well with normal selenium, but it has never worked with selenium-driverless (always opting different errors). i will provide both codes so you can try to reproduce the errors and know what's causing it.
@alimayorru Sure, I can have a look at it:)
Can you provide me the errors (including tracebacks) for the driverless script as well?
it shows different errors at different times, that's why i provided both codes for you to test and reproduce
Hello @kaliiiiiiiiii , any update on it?
@alimayorru I've got the following errors:
cdp_socket.exceptions.CDPError: {'code': -32000, 'message': 'Could not compute box model.'}
seems related to https://github.com/chromedp/chromedp/issues/94
Maybe you're using nodeId of another element which is hidden but which enters into the same selector search... It happened to me
Might implement a automatic option to use element.getBoundingClientRect()
, however, this will require TODO add Page.createIsolatedWorld
support with DOM
access first to be undetectable.
cdp_socket.exceptions.CDPError: {'code': -32000, 'message': 'Node is detached from document'}
looks like CDP isn't able to scroll to a element sometimes. I anyways plan to use Input.dispatchMouseEvent
in some future for scolling, as the current implementation kind-of teleports to itNote: clicking is different with driverless, as it doesn't require to switch to a iframe and clicks on a random location within it. Therefore you'll get different results. You can as well use https://github.com/kaliiiiiiiiii/Selenium-Driverless#pointer-interaction directly
Or have you got any other errors??
How can I use pointer without asyncio?,
How can I empliment it in my code example?
@alimayorru Pointer hasn't a implementation for sync yet.
same code works well with asyncio (using elem.click).
i don't know how to use driverless asyncio with selenium profile, can you show me an example of driverless asyncio with profile?
@alimayorru selenium-profiles
is currently synchronous only. I plan to add a async version tho:)
hello @kaliiiiiiiiii have you implemented a better way to click the enter button?
hello @kaliiiiiiiiii have you implemented a better way to click the enter button?
nope, not yet. But will be implemented with driver.send_keys("\n")
in some time
Hello, find_elements is returning a JSNodeList example : JSNodeList("XPathResult",obj_id=1664505010370382709.9.2068, context_id=9) Instead of a webelement. How can I iterate over the different elements ? Thanks
Hello, find_elements is returning a JSNodeList example : JSNodeList("XPathResult",obj_id=1664505010370382709.9.2068, context_id=9) Instead of a webelement. How can I iterate over the different elements ? Thanks
JSNodeList should just work like a regular list
I'm using the sync version and it looks like it's not iterable or empty.
I'm using the sync version and it looks like it's not iterable or empty.
well then it's probably a empty list? As long elem.__next__()
or elem.__iter__.__next__()
exist, it is iterable
i have tried finding all elements using different format (xpath, css_selector, tag_name etc) but i get error
cdp_socket.exceptions.CDPError: {'code': -32000, 'message': 'No node with given id found'}
here's my code