marty90 / PyChromeDevTools

PyChromeDevTools is a python module that allows one to interact with Google Chrome using Chrome DevTools Protocol within a Python script.
Apache License 2.0
297 stars 41 forks source link

websocket._exceptions.WebSocketConnectionClosedException: socket is already closed. #19

Open nba94 opened 5 years ago

nba94 commented 5 years ago

Hi, I open the chrome browser and try to do anything after connecting to the dev tools, but anything I try results in 'socket is already closed'...

Any suggestions?? (I have tried all of those lines and all of them end up throwing the socket closed error.. chrome = PyChromeDevTools.ChromeInterface() - seems to be working fine.


subprocess.call(['open', '-a', "/Applications/Google Chrome.app", '--args', '--remote-debugging-port=9222'])
chrome = PyChromeDevTools.ChromeInterface()
chrome.Network.enable()
# chrome.Page.enable()
# chrome.Page.navigate(url="https://www.google.com")
# chrome.wait_event("Page.loadEventFired", timeout=60000)```
tgianko commented 5 years ago

I have the very same problem here. The exception originates here:

self.parent.ws.send(json.dumps(call_obj))

Somehow (for reasons), the websocket at that point is closed. It turns out that the line of code that does something (not sure what or why) is here:

self.parent.pop_messages()

Commenting out that line solves the issue. However, I do not know if this little change will break the time-space continuum.

(Update) It turns out the exception is fired when a .recv() is done right after establishing a new connection and after a .settimeout(0). From that moment on, the websocket is unusable.

code4days commented 5 years ago

I had the same issue, whats happening is that when you create a new instance without specifying a timeout chrome = PyChromeDevTools.ChromeInterface(), the timeout for the socket defaults to 1 second here, so basically the timeout you're passing into wait_event is useless b/c message = self.ws.recv() times out within 1 second and raises an exception. To get around this specify a timeout when creating the ChromeInterface instance e.g. chrome = PyChromeDevTools.ChromeInterface(timeout=120), but now the timeout you pass into wait_event is only effective if its less than the socket timeout and there's data coming in through the socket or else it will wait till the socket times out.

So the timeout in wait_event essentially means wait for the passed in event to show up in incoming messages until we hit the timeout, if no data is coming in the socket will timeout within a second, unless specified when you initialize ChromeInterface.

marty90 commented 5 years ago

Guys I cannot reproduce the error. It is working fine on my Ubuntu and MacOS PCs. Do you have any special configuration of Chrome?

mylh commented 5 years ago

I have the same issue on only 1 server among ~20 the code runs at. Tried to specify timeout on ChromeInterface - didn't work. So this seems to be specific to server configuration. It is failing on linode instance, but works perfectly on digitalocean and vultr and on my local dev machines. All instances has equal CPU and RAM. The traceback is as follows:

chrome.Network.enable()
File "/home/webapps/local/lib/python2.7/site-packages/PyChromeDevTools/__init__.py", line 27, in generic_function
self.parent.ws.send(json.dumps(call_obj))
File "/home/webapps/local/lib/python2.7/site-packages/websocket/_core.py", line 253, in send
return self.send_frame(frame)
File "/home/webapps/local/lib/python2.7/site-packages/websocket/_core.py", line 278, in send_frame
l = self._send(data)
File "/home/webapps/local/lib/python2.7/site-packages/websocket/_core.py", line 448, in _send
return send(self.sock, data)
File "/home/webapps/local/lib/python2.7/site-packages/websocket/_socket.py", line 132, in send
raise WebSocketConnectionClosedException("socket is already closed.")
WebSocketConnectionClosedException: socket is already closed.
nba94 commented 5 years ago

I have tried thechrome = PyChromeDevTools.ChromeInterface(timeout=120) suggestion but still does not work. I am using the latest chrome version, nothing is custom. Also I have tried the same thing on 2 different MacOS machines (Air and Pro) - the issue stands the same.

Below is the code I am trying to run:

import threading
import os
import subprocess
import PyChromeDevTools
import time

def open_chrome_supreme():
    subprocess.call(['open', '-a', "/Applications/Google Chrome.app", '--args', '--remote-debugging-port=9222'])

t = threading.Thread(target=open_chrome_supreme)
t.start()

time.sleep(10)

chrome = PyChromeDevTools.ChromeInterface(timeout=10000)
chrome.Page.navigate(url="https://www.google.com")
chrome.wait_event("Page.loadEventFired", timeout=60000)

And this is what I get when I try running it:

Traceback (most recent call last):
  File "test1.py", line 16, in <module>
    chrome.Page.navigate(url="https://www.google.com")
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PyChromeDevTools/__init__.py", line 27, in generic_function
    self.parent.ws.send(json.dumps(call_obj))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/websocket/_core.py", line 253, in send
    return self.send_frame(frame)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/websocket/_core.py", line 278, in send_frame
    l = self._send(data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/websocket/_core.py", line 448, in _send
    return send(self.sock, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/websocket/_socket.py", line 132, in send
    raise WebSocketConnectionClosedException("socket is already closed.")
websocket._exceptions.WebSocketConnectionClosedException: socket is already closed.
Freakazo commented 5 years ago

@marty90 can you try and update so that websocket-client==0.55.0 which was release not too long ago to see if you can replicate the issue then?

I restricted the version to websocket-client==0.54.0 and the error others are getting above disappeared. @nba94 @mylh can youse force install that version of websocket-client to see if youse can still replicate the issue?

nba94 commented 5 years ago

@Freakazo I have force installed websocket-client==0.54.0 and it seems the issue is gone now! I will do some further testing in my main script and will update my comment.

@marty90 @mylh

mylh commented 5 years ago

Hi guys. I can confirm that installing websocket-client==0.54.0 has solved the issue for me. Thanks!