ClericPy / ichrome

Chrome controller for Humans, based on Chrome Devtools Protocol(CDP) and python3.7+.
https://pypi.org/project/ichrome/
MIT License
228 stars 29 forks source link

sync querySelectorAll #64

Closed ttygde closed 2 years ago

ttygde commented 2 years ago
browser = Chrome(host="127.0.0.1", port=9222, timeout=3, retry=1)
tab = self.br.tabs[0]
tab.set_url('https://bing.com', timeout=5)
import time
time.sleep(3)
sel = tab.querySelectorAll('#sb_form_q')
INFO  2021-10-09 21:27:08 [ichrome] sync_utils.py(247): <Tab(chrome://newtab/)> send: {'method': 'Page.navigate', 'params': {'url': 'https://bing.com'}, 'id': 1}
INFO  2021-10-09 21:27:18 [ichrome] sync_utils.py(489): querySelectorAll crashed for: TypeError('the JSON object must be str, bytes or bytearray, not FailureException'), response: FailureException: TimeoutError()

File "/home/j/.local/lib/python3.9/site-packages/ichrome/sync_utils.py", line 492, in querySelectorAll
    raise error
  File "/home/j/.local/lib/python3.9/site-packages/ichrome/sync_utils.py", line 478, in querySelectorAll
    response = json.loads(response)["result"]["result"]["value"]
  File "/usr/lib/python3.9/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not FailureException
ClericPy commented 2 years ago

TimeoutError means your Chrome debugging mode is not ready (or the connection is bad). You should launch the chrome daemon with python3 -m ichrome at first.

ttygde commented 2 years ago

It loads bing.com but cannot find any element on it. Chrome daemon is running.

ClericPy commented 2 years ago

Ensure http://127.0.0.1:9222 is available. The Exception is Timeout so it may be raised by a bad connection.

sync mode is deprecated, I try your code but it all works(I changed self.br to browser ).

ttygde commented 2 years ago

thanks, problem was on my side

ttygde commented 2 years ago

Finally I decided to switch to async version. It is such a gem. Thanks for this wonderful project!

ClericPy commented 2 years ago

Finally I decided to switch to async version. It is such a gem. Thanks for this wonderful project!

Thanks for your Thanks!

If you really don't like asyncio, you may try the playwright from https://github.com/microsoft/playwright-python

ichrome is only a personal product, I used it on my daily jobs at work but only fixed the bugs of async mode.

But playwright and pyppeteer are developped by official organizations, the playwright support both sync and async mode. The only reason I quit them is they were too late or too confused for me.

ttygde commented 2 years ago

Playwright is too heavy and popular thus more easily detectable by sites. ichrome is low-profile, simple and nice, exactly what I was looking for.

ClericPy commented 2 years ago

Alright... I thought they are the same usage of Chrome CDP yet. Sometimes I just want to write a pure Websocket connection manager for CDP...