d-Rickyy-b
commented
9 months ago Hi there, thanks for creating the issue.
The first thing I noticed is you use wss:// locally. Are you actually using certificates? Otherwise just use ws://
You seem to be using the domains-only endpoint (which is good when you only need domains).
certstream.listen_for_events(print_callback, url='wss://192.168.1.101/domains-only/')
But at the same time in your first example you try to fetch the certs from the json path data.leaf_cert.all_domains, which does not exist for domains-only! See here for the code that generates the domains-only json.
all_domains = message['data']['leaf_cert']['all_domains']
You can always fetch an example cert from certstream-server-go to inspect the json format by using http and appending /example.json to the endpoint's name. In your case it would be http://192.168.1.101/domains-only/example.json. Or for debugging you can also check on my hosted instance:
https://certstream.rico-j.de/domains-only/example.json (this hosted instance might go offline at any time. It is just there for debugging and testing purposes.)
The json looks like this:
{
"data": [
"*.oleksandrtsvietkov.direct.quickconnect.to",
"oleksandrtsvietkov.direct.quickconnect.to"
],
"message_type": "dns_entries"
}So you should access the domains like this:
all_domains = message['data']
Also please check the server's log for indicators that your client is too slow. Python is quite a slow language and my server processes around 300-400 certificates per second (!). People noticed in other issues (https://github.com/d-Rickyy-b/certstream-server-go/issues/28) that their code was simply too slow to process all of the certificates (although they processed the whole certificates, not the domains only). I highly suggest using multiple threads.
Please let me know if any of this helped.
cipisek9
Hello, I have started using your certstream server, but I have the problem. I have python script which is reading stream (endpoint domain-only) and looking for domains from my defined list. Unfortunately a lot of domains are missing in compare to list of catched domains from calidog certstream.
I have two solutions, but neither of them works:
First solution is the same script, as I am using for calidog certstream. With their stream works fine, with my own (your solution) not very well
Second solution - same results as with previous script, domains are missing
Could someone help me, how to find where is the problem, and how can I fix this? Thank you in advance.