smicallef / spiderfoot

SpiderFoot automates OSINT for threat intelligence and mapping your attack surface.
http://www.spiderfoot.net
MIT License
13.08k stars 2.27k forks source link

Unicode Error Break one scan and all after it #1636

Open Esonhugh opened 2 years ago

Esonhugh commented 2 years ago

Unhandled exception (UnicodeError) encountered during scan.Please report this as a bug: ['Traceback (most recent call last):\n', ' File "/usr/lib/python3.8/encodings/idna.py", line 181, in encode\n result.extend(ToASCII(label))\n', ' File "/usr/lib/python3.8/encodings/idna.py", line 76, in ToASCII\n label = nameprep(label)\n', ' File "/usr/lib/python3.8/encodings/idna.py", line 38, in nameprep\n raise UnicodeError("Invalid character %r" % c)\n', "UnicodeError: Invalid character '\x8f'\n", '\nThe above exception was the direct cause of the following exception:\n\n', 'Traceback (most recent call last):\n', ' File "/root/spiderfoot/sfscan.py", line 392, in startScan\n if self.sf.isDomain(self.targetValue, self.config[\'_internettlds\']):\n', ' File "/root/spiderfoot/sflib.py", line 830, in isDomain\n ps = PublicSuffixList(tldList, only_icann=True, accept_unknown=False)\n', ' File "/usr/local/lib/python3.8/dist-packages/publicsuffixlist/init.py", line 75, in init\n self._parse(source, accept_encoded_idn, only_icann=only_icann)\n', ' File "/usr/local/lib/python3.8/dist-packages/publicsuffixlist/init.py", line 108, in _parse\n e = encode_idn(s.lstrip("!"))\n', ' File "/usr/local/lib/python3.8/dist-packages/publicsuffixlist/init.py", line 42, in encode_idn\n return u(domain).encode("idna").decode("ascii")\n', "UnicodeError: encoding with 'idna' codec failed (UnicodeError: Invalid character '\x8f')\n"]

python version 3.8

run failure at once happened.

smicallef commented 2 years ago

Could you please share the target, if possible?

Esonhugh commented 2 years ago

Could you please share the target, if possible?

first is homeboyc.cn next is eson.ninja

9je commented 2 years ago

Encountered same error as well, my first scan (an ip) worked fine, but every following scan (including if i try rerunning the once previous successful scan) returns "pythonlocation\lib\encodings\cp1252.py" ,line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 10466: character maps to <undefined>

JCubeSVK commented 1 year ago

Has there been any fix to this? Encountering the same issue.