search

CaliDog / certstream-python

Python library for connecting to CertStream
MIT License
426 stars 72 forks source link

Experiencing frequent websocket stream reconnects #27

Closed lorenzbausch closed 4 years ago

lorenzbausch commented 5 years ago

Using version 1.10 and the sample code from the readme, roughly every minute a reconnect occurs, preceded by the following message:

[ERROR:root] 2019-10-11 16:38:18,582 - Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
0xroot-bf commented 4 years ago

I've experienced the same issue here. At first thought it could be associated with WSS on low ports, but even when running any example as root, still hits the same issue.

This is happening using the following requirements:

certstream==1.10
requests
ipwhois
PySocks
hues
websocket-client==0.48.0
apprise
neonknight commented 4 years ago

same for me, not getting any certstream data, only reconnects. also running

certstream==1.10
websocket-client==0.48.0
Fitblip commented 4 years ago

Hey folks, sorry for the delay here.

Digging into this, it looks like cowboy (the underlying websocket server for certstream) will timeout connections after a minute and force reconnection - https://github.com/ninenines/cowboy/issues/1259

I didn't realize this and figured another mechanism would be handling heartbeats, which doesn't seem to be the case. I'm going to add heartbeating to this and the other clients which should (hopefully) remedy these timeout issues (the golang client is experiencing this as well - https://github.com/CaliDog/certstream-go/issues/8).

Fitblip commented 4 years ago

This is now live with certstream-python==1.11! Let me know if you have any issues.

neonknight commented 4 years ago

looks good so far, thank you for the fix!

0xroot-bf commented 4 years ago

thanks for the fix, works like charm!

t4d commented 4 years ago

Nice! Thank you for the fix.

Fitblip commented 4 years ago

Thanks for the info! Cheers, and happy holidays!