[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x6b096b]

amalmurali47 commented 6 years ago

What's the problem (or question)?

Attempting to run it for yahoo.com generates some errors.

I don't believe I'm having any internet issues on my computer. Other scripts that pull data from the web work fine. DNS resolution is working. No firewalls.

Do you have an idea for a solution?

Errors should be handled gracefully. The user should not see the traceback etc. They should be logged, and the user should be shown a friendly error message saying why it failed.

How can we reproduce the issue?

I did the following:

./subfinder -d yahoo.com

Got this response:

[-] Searching For Subdomains in Ask
[-] Searching For Subdomains in Baidu
[-] Searching For Subdomains in Bing
[-] Searching For Subdomains in Censys
[-] Searching For Subdomains in Crt.sh
[-] Searching For Subdomains in CertDB
[-] Searching For Subdomains in Certspotter
[-] Searching For Subdomains in Dnsdb
[-] Searching For Subdomains in Threatcrowd
[-] Searching For Subdomains in Findsubdomains
[-] Searching For Subdomains in DNSDumpster
[-] Searching For Subdomains in PassiveTotal
[-] Searching For Subdomains in PTRArchive
[-] Searching For Subdomains in Hackertarget
[-] Searching For Subdomains in Virustotal
[-] Searching For Subdomains in Securitytrails
[-] Searching For Subdomains in WaybackArchive
[-] Searching For Subdomains in ThreatMiner
[-] Searching For Subdomains in Riddler
[-] Searching For Subdomains in Netcraft

[+] Finding subdomains for : yahoo.com
error: Get https://www.threatcrowd.org/searchApi/v2/domain/report/?domain=yahoo.com: dial tcp [2400:cb00:2048:1::681f:b9a]:443: connect: network is unreachable

error: Get https://www.bing.com/search?q=domain%3Ayahoo.com&go=Submit&first=0: dial tcp: lookup www.bing.com on 127.0.0.53:53: read udp 127.0.0.1:47844->127.0.0.53:53: i/o timeout

error: Get http://ptrarchive.com/tools/search2.htm?label=yahoo.com&date=ALL: dial tcp: lookup ptrarchive.com on 127.0.0.53:53: read udp 127.0.0.1:45089->127.0.0.53:53: i/o timeout

error: Get http://www.dnsdb.org/f/yahoo.com.dnsdb.org/: dial tcp [2400:cb00:2048:1::681f:5769]:80: connect: network is unreachable

error: Get https://www.threatminer.org/getData.php?e=subdomains_container&q=yahoo.com&t=0&rt=10&p=1: dial tcp [2400:cb00:2048:1::6819:b105]:443: connect: network is unreachable

error: Get https://api.hackertarget.com/hostsearch/?q=yahoo.com: dial tcp: lookup api.hackertarget.com on 127.0.0.53:53: read udp 127.0.0.1:56787->127.0.0.53:53: i/o timeout

error: Get https://certdb.com/domain/yahoo.com: dial tcp: lookup certdb.com on 127.0.0.53:53: read udp 127.0.0.1:36248->127.0.0.53:53: i/o timeout

error: Get https://certspotter.com/api/v0/certs?domain=yahoo.com: dial tcp: lookup certspotter.com on 127.0.0.53:53: read udp 127.0.0.1:38982->127.0.0.53:53: i/o timeout
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x6b096b]

goroutine 21 [running]:
github.com/Ice3man543/subfinder/libsubfinder/sources/riddler.Query(0xc4200ba200, 0xc420096420)
    /home/user/go/src/github.com/Ice3man543/subfinder/libsubfinder/sources/riddler/riddler.go:56 +0x24b
created by github.com/Ice3man543/subfinder/libsubfinder/engines/passive.Discover
    /home/user/go/src/github.com/Ice3man543/subfinder/libsubfinder/engines/passive/passive.go:151 +0xb3b

What are the running context details?

Installation method (e.g. pip, apt-get, git clone or zip/tar.gz): go get
Client OS (e.g. Microsoft Windows 10): Ubuntu 18.04
Program version (see banner): v0.1.0
Relevant console output (if any): See above.
Exception traceback (if any): See above.

Ice3man543 commented 6 years ago

Most probably, your internet is not working correctly with SubFinder. Will look at it in a moment.

amalmurali47 commented 6 years ago

It seems to be working now. Maybe it was some connectivity issue. Please look into this.

On a side note, I believe it should handle the errors more gracefully. Suggestions:

Before attempting to start getting the data from different sources, ensure that the Internet is working fine, and proceed only if it is working. This could include checking for the response codes etc. from the websites. It would help if the output said which sources did not get any data and why.
If some errors are encountered, they must be logged. Users should only see the basic error messages - just the bare minimum. They don't really want the tracebacks etc. If they need more detailed output, they can use the verbose mode. Or check the logs.

What are your thoughts?

Ice3man543 commented 6 years ago

Yeah, you're right. I was pretty busy till now, so didn't had time to implement it. The ideas are good. Will implement them soon.

projectdiscovery / subfinder