search

devanshbatham / CertEagle

Weaponizing Live CT logs for automated monitoring of assets
133 stars 36 forks source link

Error connecting to CertStream #2

Closed joeldeleep closed 4 years ago

joeldeleep commented 4 years ago

I am getting this error as shown below, is this intended behaviour ? 

Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...

In between the error lines if a match is found it is pushed to slack . When I checked with the help of certstream package , I got the below output. 

certstream 
[INFO:certstream] 2020-08-11 10:54:44,338 - Connection established to CertStream! Listening for events...
[2020-08-11T10:54:44.328468] ct.googleapis.com/logs/argon2021/ - *.admin-eu2.cas.ms
[2020-08-11T10:54:44.327114] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.326011] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.324387] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.322587] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.320814] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.318968] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.317324] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.315464] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.313734] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
devanshbatham commented 4 years ago

Hello,

The information you provided seems incomplete, provide me the details of the following :

I tested this on Azure VPS and it's running from past 2 months continuesly, working for me !

I am into this anyways, will let you know if I will find the root cause

Thanks

joeldeleep commented 4 years ago

Hai there ,

  1. VPS - Digital Ocean 2.Connection is stable , I am running other tools and scripts without issues 3.Ubuntu 20.04

As I mentioned int the issue , the certsream command is working without any problems . I think maybe filtering the domains are causing the error , since there is no match and when the match is there it is pushed to slack .

am6539 commented 4 years ago

same problem

devanshbatham commented 4 years ago

Hello

After digging into this I came to know that the error message is emitted by websocket-client package

Note : Yes you will get these errors in between sometimes as the websocket connection will get closed and connected again , that is expected behavior , But if you are only getting these errors and no output then let me know !

I am looking for a permanent fix, I will push the changes as soon I will figure out something

rgds Devansh

joeldeleep commented 4 years ago

Kindly note the below error for certstream

ERROR: certstream 1.11 has requirement websocket-client>=0.56.0, but you'll have
 websocket-client 0.48.0 which is incompatible.
devanshbatham commented 4 years ago

Hello,

Turns the problem is more tricky then speculated, I am looking for a fix , and will update if any success !

Thanks

nickjenkins83 commented 4 years ago

Change requirements.txt to show the same as I have. certstream==1.10 websocket-client==0.48.0

It will work after this. Went from not working to working with just this change.

devanshbatham commented 4 years ago

Hello,

I pushed the changes! Hopefully, it should work fine now

Let me know if the error persists.

rgds Devansh

joeldeleep commented 4 years ago

I think its ok now , I will update if any issue is there .