search

psf / requests-html

Pythonic HTML Parsing for Humans™
http://html.python-requests.org
MIT License
13.64k stars 977 forks source link

Websocket failing connection when rendering #570

Open Nabstar3 opened 3 months ago

Nabstar3 commented 3 months ago

Hello,

This is my first time installing and using requests_html.

When attempting to render a webpage using the .render method, a websocket which I'm assuming connects to the local chromium installation fails. 

[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:asyncio] Using proactor: %s
[D:websockets.client] = connection is CONNECTING
[D:websockets.client] > GET %s HTTP/1.1
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] < HTTP/1.1 %d %s
[D:websockets.client] < %s: %s
[D:websockets.client] < %s: %s
[D:websockets.client] < %s: %s
[D:websockets.client] < %s: %s
[D:websockets.client] = connection is OPEN
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] > %s
[D:websockets.client] > %s
[D:websockets.client] > %s
[D:websockets.client] > %s
[D:websockets.client] > %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] > %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] = connection is CLOSING
[D:websockets.client] > %s
[D:websockets.client] ! failing connection with code %d
[D:websockets.client] = connection is CLOSED

In addition, the first time rendering failed to get the chromium installation package, hence I referred to here to solve that by installing an older version of pyppeteer.

I also reffered to #569 during initial run.

RonCodes88 commented 1 month ago

I have the same issue, do you know the fix?

dereklee0310 commented 2 weeks ago

Same issue here, any update?

OskarFdz11 commented 4 days ago

Same issue, somebody knows the solution?

RonCodes88 commented 4 days ago

What are you trying to achieve? I might know the fix.

OskarFdz11 commented 4 days ago

Hi, i'm trying to extract the information from these products through web scraping with python, in this link i have send my question on stack overflow: https://stackoverflow.com/questions/78673625/python-web-scraping-dwebsockets-client-get-s-http-1-1-dwebsockets-client

Basically i want to extract the information of these web page products: https://bodysolid-europe.com/collections/all But in the console just appears the message:

D:websockets.client] < %s [D:websockets.client] < %s [D:websockets.client] < %s [D:websockets.client] < %s [D:websockets.client] < %s [D:websockets.client] < %s [D:websockets.client] < %s [D:websockets.client] < %s

El mié, 26 jun 2024 a la(s) 1:27 p.m., Ronald Li @.***) escribió:

What are you trying to achieve? I might know the fix.

— Reply to this email directly, view it on GitHub https://github.com/psf/requests-html/issues/570#issuecomment-2192476413, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYCZ7EHEMQ3POXD6XIAOKKLZJMI3RAVCNFSM6AAAAABFS56UROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSGQ3TMNBRGM . You are receiving this because you commented.Message ID: @.***>

RonCodes88 commented 4 days ago

I see. Firstly, you should double-check you are correctly scraping the link and the information you need. I didn't end up figuring out the issue, but I had a workaround, which is to use selenium web driver. Let me know if you figured it out.