wummel / linkchecker

check links in web documents or full websites
http://wummel.github.io/linkchecker/
GNU General Public License v2.0
1.41k stars 233 forks source link

You have found an internal error in LinkChecker. #723

Open AtomXY opened 7 years ago

AtomXY commented 7 years ago

Scanning the url from command line linkchecker.exe --user-agent "LinkChecker spider" -t20 -a -r10 --file-output=csv/utf_8/Report.csv https://www.mql5.com/

[19:18:35][MQL5_LinkCheck] ** Oops, I did it again. ***** [19:18:35][MQL5_LinkCheck] [19:18:35][MQL5_LinkCheck] You have found an internal error in LinkChecker. Please write a bug report [19:18:35][MQL5_LinkCheck] at https://github.com/wummel/linkchecker/issues [19:18:35][MQL5_LinkCheck] and include the following information: [19:18:35][MQL5_LinkCheck] - the URL or file you are testing [19:18:35][MQL5_LinkCheck] - the system information below [19:18:35][MQL5_LinkCheck] [19:18:35][MQL5_LinkCheck] When using the commandline client: [19:18:35][MQL5_LinkCheck] - your commandline arguments and any custom configuration files. [19:18:35][MQL5_LinkCheck] - the output of a debug run with option "-Dall" [19:18:35][MQL5_LinkCheck] [19:18:35][MQL5_LinkCheck] Not disclosing some of the information above due to privacy reasons is ok. [19:18:35][MQL5_LinkCheck] I will try to help you nonetheless, but you have to give me something [19:18:35][MQL5_LinkCheck] I can work with ;) . [19:18:35][MQL5_LinkCheck] [19:18:35][MQL5_LinkCheck] Traceback (most recent call last): [19:18:35][MQL5_LinkCheck] File "linkcheck\director\checker.pyo", line 104, in check_url [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\director\checker.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\director\checker.pyo", line 120, in check_url_data [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\director\checker.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\director\checker.pyo", line 53, in check_url [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\director\checker.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\checker\urlbase.pyo", line 465, in check_content [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\checker\urlbase.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\checker\urlbase.pyo", line 555, in allows_recursion [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\checker\urlbase.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\checker\httpurl.pyo", line 83, in content_allows_robots [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\checker\httpurl.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\checker\urlbase.pyo", line 611, in get_content [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\checker\urlbase.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] File "linkcheck\checker\httpurl.pyo", line 304, in read_content [19:18:35][MQL5_LinkCheck] -- couldn't find file, trying this instead: c:\cdn\linkchecker\library.zip\linkcheck\checker\httpurl.py [19:18:35][MQL5_LinkCheck] -- code not available -- [19:18:35][MQL5_LinkCheck] MemoryError [19:18:38][MQL5_LinkCheck] System info: [19:18:38][MQL5_LinkCheck] LinkChecker 9.3 [19:18:38][MQL5_LinkCheck] Released on: 16.7.2014 [19:18:38][MQL5_LinkCheck] Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win32 [19:18:38][MQL5_LinkCheck] Requests: 2.2.1 [19:18:38][MQL5_LinkCheck] Modules: Sqlite [19:18:38][MQL5_LinkCheck] Local time: 2017-06-22 19:18:30+002 [19:18:38][MQL5_LinkCheck] sys.argv: ['c:\cdn\linkchecker\linkchecker.exe', '--user-agent', 'LinkChecker spider', '-t20', '-a', '-r10', '--file-output=csv/utf_8/Report.csv', 'https://www.mql5.com/'] [19:18:38][MQL5_LinkCheck] Default locale: ('en', 'cp1251') [19:18:38][MQL5_LinkCheck] [19:18:38][MQL5_LinkCheck] [19:18:38][MQL5_LinkCheck] **** LinkChecker internal error, over and out **** [19:18:38][MQL5_LinkCheck] WARNING 2017-06-22 19:18:30,845 CheckThread-https://www.mql5.com/ru/market/mt4/page127 internal error occurred [19:18:38][MQL5_LinkCheck] [19:18:38][MQL5_LinkCheck] WARNING 2017-06-22 19:18:30,846 CheckThread-https://www.mql5.com/ru/market/mt4/page127 internal error occurred

SilkAndSlug commented 7 years ago

Same problem. The final two tracebacks are (oldest first):

File "/usr/lib/python2.7/dist-packages/linkcheck/checker/httpurl.py", line 193, in _add_ssl_info line: sock = self._get_ssl_sock() locals: sock = self = <https link, base_url=u'/./Home', parent_url=u'https://www.edithwilmot.co.uk', base_ref=None, recursion_level=1, url_connection=None, line=35, column=22, page=0, name=u'Home', anchor=u'', cache_url=https://www.edithwilmot.co.uk/Home> self._get_ssl_sock = <bound method HttpUrl._get_ssl_sock of <https link, base_url=u'/./Home', parent_url=u'https://www.edithwilmot.co.uk', base_ref=None, recursion_level=1, url_connection=None, line=35, column=22, page=0, name=u'Home', anchor=u'', cache_url=https://www.edithwilmot.co.uk/Home>>

File "/usr/lib/python2.7/dist-packages/linkcheck/checker/httpurl.py", line 184, in _get_ssl_sock line: if raw_connection.sock is None: locals: raw_connection = None raw_connection.sock = !AttributeError: 'NoneType' object has no attribute 'sock' None = None AttributeError: 'NoneType' object has no attribute 'sock'

My system info is:

System info: LinkChecker 9.3 Released on: 16.7.2014 Python 2.7.9 (default, Jun 29 2016, 13:08:31) [GCC 4.9.2] on linux2 Requests: 2.17.3 Modules: Sqlite Local time: 2017-06-18 00:00:08+001 sys.argv: ['/usr/bin/linkchecker', '-f', '/home/silkandslug/www/edithwilmot/dev/../linkchecker/public.rc', 'https://www.edithwilmot.co.uk'] LANGUAGE = 'en_GB:en' LANG = 'en_GB.UTF-8' Default locale: ('en', 'UTF-8')

Config file is attached

dpalic commented 6 years ago

Thank you for the issue report. Sadly this project is dead, and a new team is around with https://github.com/linkcheck/linkchecker for more details please see: #708 Also please close this issue and report it freshly on the new repo https://github.com/linkcheck/linkchecker/issues