search

PrivacyScore / privacyscanner

Privacyscanner is the new scanning backend component of a future PrivacyScore version
MIT License
20 stars 11 forks source link

Traceback when scan module `mail` failed for Network is unreachable #25

Open jayvdb opened 4 years ago

jayvdb commented 4 years ago

The backtrace doesnt give much information which the error message already provides, and occurs several times. If it is needed, it should be at log level debug or at least info.

> privacyscanner scan http://blog.girlscouts.org
chromedevtools> 2020-02-20 19:07:50,176: [INFO] Starting chromedevtools (scanner.py:180)
chromedevtools> 2020-02-20 19:08:33,356: [INFO] Finished chromedevtools (scanner.py:204)
dns> 2020-02-20 19:08:33,357: [INFO] Starting dns (scanner.py:180)
dns> 2020-02-20 19:08:33,359: [WARNING] GeoIP database not available. Country lookup disabled. (dns.py:173)
dns> 2020-02-20 19:08:33,556: [WARNING] [] (dns.py:49)
dns> 2020-02-20 19:08:33,837: [INFO] Finished dns (scanner.py:204)
serverleaks> 2020-02-20 19:08:33,838: [INFO] Starting serverleaks (scanner.py:180)
unable to cache TLDs in file /usr/lib/python3.7/site-packages/tldextract/.tld_set: [Errno 13] Permission denied: '/usr/lib/python3.7/site-packages/tldextract/.tld_set'
serverleaks> 2020-02-20 19:08:43,193: [INFO] Finished serverleaks (scanner.py:204)
mail> 2020-02-20 19:08:43,195: [INFO] Starting mail (scanner.py:180)
mail> 2020-02-20 19:08:43,199: [WARNING] No CA certificates loaded. Cannot check for trust. (mail.py:52)
mail> 2020-02-20 19:08:53,312: [ERROR] Scan module `mail` failed. (scanner.py:198)
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanner.py", line 185, in scan_site
    mod.scan_site(result, scan_meta)
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanmodules/mail.py", line 67, in scan_site
    code, banner = conn.connect(mail_host)
  File "/usr/lib64/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/lib64/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib64/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable
mail> 2020-02-20 19:08:53,345: [INFO] Finished mail (scanner.py:204)
mail> 2020-02-20 19:08:53,345: [INFO] Starting mail (scanner.py:180)
mail> 2020-02-20 19:08:53,347: [WARNING] No CA certificates loaded. Cannot check for trust. (mail.py:52)
mail> 2020-02-20 19:09:03,360: [ERROR] Scan module `mail` failed. (scanner.py:198)
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanner.py", line 185, in scan_site
    mod.scan_site(result, scan_meta)
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanmodules/mail.py", line 67, in scan_site
    code, banner = conn.connect(mail_host)
  File "/usr/lib64/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/lib64/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib64/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable
mail> 2020-02-20 19:09:03,386: [INFO] Finished mail (scanner.py:204)
mail> 2020-02-20 19:09:03,387: [INFO] Starting mail (scanner.py:180)
mail> 2020-02-20 19:09:03,389: [WARNING] No CA certificates loaded. Cannot check for trust. (mail.py:52)
mail> 2020-02-20 19:09:13,401: [ERROR] Scan module `mail` failed. (scanner.py:198)
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanner.py", line 185, in scan_site
    mod.scan_site(result, scan_meta)
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanmodules/mail.py", line 67, in scan_site
    code, banner = conn.connect(mail_host)
  File "/usr/lib64/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/lib64/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib64/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable
mail> 2020-02-20 19:09:13,430: [INFO] Finished mail (scanner.py:204)
mail> 2020-02-20 19:09:13,431: [INFO] Starting mail (scanner.py:180)
mail> 2020-02-20 19:09:13,433: [WARNING] No CA certificates loaded. Cannot check for trust. (mail.py:52)
mail> 2020-02-20 19:09:23,446: [ERROR] Scan module `mail` failed. (scanner.py:198)
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanner.py", line 185, in scan_site
    mod.scan_site(result, scan_meta)
  File "/usr/lib/python3.7/site-packages/privacyscanner/scanmodules/mail.py", line 67, in scan_site
    code, banner = conn.connect(mail_host)
  File "/usr/lib64/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/lib64/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/lib64/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable
mail> 2020-02-20 19:09:23,474: [INFO] Finished mail (scanner.py:204)
testssl_https> 2020-02-20 19:09:23,474: [INFO] Starting testssl_https (scanner.py:180)
testssl_https> 2020-02-20 19:09:23,476: [INFO] Current stage: basic (base.py:66)
testssl_https> 2020-02-20 19:12:52,459: [INFO] testssl.sh result is complete. (base.py:97)
testssl_https> 2020-02-20 19:12:52,460: [INFO] Next stage: vulns (base.py:119)
testssl_https> 2020-02-20 19:12:52,487: [INFO] Finished testssl_https (scanner.py:204)
testssl_https> 2020-02-20 19:13:02,501: [INFO] Starting testssl_https (scanner.py:180)
testssl_https> 2020-02-20 19:13:02,503: [INFO] Current stage: vulns (base.py:66)
testssl_https> 2020-02-20 19:14:00,578: [INFO] testssl.sh result is complete. (base.py:97)
testssl_https> 2020-02-20 19:14:00,579: [INFO] Next stage: vulns_ids (base.py:119)
testssl_https> 2020-02-20 19:14:00,609: [INFO] Finished testssl_https (scanner.py:204)
testssl_https> 2020-02-20 19:14:10,623: [INFO] Starting testssl_https (scanner.py:180)
testssl_https> 2020-02-20 19:14:10,624: [INFO] Current stage: vulns_ids (base.py:66)
testssl_https> 2020-02-20 19:15:11,949: [INFO] testssl.sh result is complete. (base.py:97)
testssl_https> 2020-02-20 19:15:12,720: [INFO] vulns_ids was the final stage. (base.py:117)
testssl_https> 2020-02-20 19:15:12,752: [INFO] Finished testssl_https (scanner.py:204)
testssl_mail> 2020-02-20 19:15:12,753: [INFO] Starting testssl_mail (scanner.py:180)
testssl_mail> 2020-02-20 19:15:12,754: [INFO] Skipping testssl.sh checks: No (START)TLS found. (base.py:53)
testssl_mail> 2020-02-20 19:15:12,811: [INFO] Finished testssl_mail (scanner.py:204)
jayvdb commented 4 years ago

I get the same for

privacyscanner scan https://blog.USNI.org/about

but not for

privacyscanner scan https://USNI.org/about

maybe it is related to being a subdomain?

Re-running on > privacyscanner scan http://girlscouts.org and no problems occur.

hprid commented 4 years ago

ENETUNREACH (errno 101) usually means that you have some problems with your local network that might need your attention, so a warning should be the proper error level. Unfortunately, I cannot reproduce the issue with the mentioned domains. The exception, however, should be cought and turned into a logger warning; I will do that within the next days.