Conso1eCowb0y / Deepminer

Deep web crawler and search engine
43 stars 10 forks source link

Kali Linux 2022.4 and Deepminer failure #1

Closed macgreine closed 1 year ago

macgreine commented 1 year ago

Deepminer worked in Kali Linux 2022.3 . I did a fresh install of Kali Linus 2022.4 loaded Deepminer with the following mess of failure messages. Any ideas? Traceback (most recent call last): File "/usr/lib/python3.10/urllib/request.py", line 1348, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/usr/lib/python3.10/http/client.py", line 1282, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output self.send(msg) File "/usr/lib/python3.10/http/client.py", line 975, in send self.connect() File "/usr/lib/python3.10/http/client.py", line 1454, in connect self.sock = self._context.wrap_socket(self.sock, File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.10/ssl.py", line 1071, in _create self.do_handshake() File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/kali/Apps/Deepminer/deepminer.py", line 6, in inputList = inputList() File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 155, in inputList ahmiaLinks = ahmia() File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 48, in ahmia with request.urlopen(req) as response: File "/usr/lib/python3.10/urllib/request.py", line 216, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.10/urllib/request.py", line 519, in open response = self._open(req, data) File "/usr/lib/python3.10/urllib/request.py", line 536, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain result = func(*args) File "/usr/lib/python3.10/urllib/request.py", line 1391, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/usr/lib/python3.10/urllib/request.py", line 1351, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)>

rlfree commented 1 year ago

Hi macgreine,

I originally wrote this in Python 3.7 and it looks like Kali is trying to run it in Pytgon 3.10. I suspect there's some sort of issue between the versions of Python, in patricular urllib library. Can you try running with Python 3.7 and report back?

macgreine commented 1 year ago

Hi, I am relatively new to Kali Linux.. I see two versions of Python in the 2023.1 release Python 11 and Python 2.7. Could you walk me through the steps to install 3.7 without screwing everything else up? By the way no joy with command Python2 or Python2.7 just got error message Traceback (most recent call last): File "deepminer.py", line 1, in from modules.deephelpers import * ImportError: No module named modules.deephelpers

rlfree commented 1 year ago

Sure, it's pretty straightforward to install Python 3.7. Here's a guide: https://tecadmin.net/install-python-3-7-on-ubuntu-linuxmint/

Python 3.7 will no longer receive security updates after June 2023, so eventually Deepminer will need to be ported over to a newer version of Python. I have no real plans on when or if I'll do that.

macgreine commented 1 year ago

I have installed Python 3.7 confirmed by -V now I suppose the command is python3.7 deepminer.py this yields Traceback (most recent call last): File "deepminer.py", line 1, in from modules.deephelpers import * File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 2, in import os, csv, re, sys, json, subprocess, socks, httplib2 ModuleNotFoundError: No module named 'socks' Any help greatly appreciated

rlfree commented 1 year ago

Sure! Just run: pip3 install pysocks

macgreine commented 1 year ago

pip3 install pysocks Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: pysocks in /usr/lib/python3/dist-packages (1.7.1) Still having the following: Traceback (most recent call last): File "deepminer.py", line 1, in from modules.deephelpers import * File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 2, in import os, csv, re, sys, json, subprocess, socks, httplib2 ModuleNotFoundError: No module named 'socks'

rlfree commented 1 year ago

You might need to run it as pip3.7 install pysocks Or python3.7 -m pip install pysocks

macgreine commented 1 year ago

Done and Traceback (most recent call last): File "deepminer.py", line 1, in from modules.deephelpers import * File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 2, in import os, csv, re, sys, json, subprocess, socks, httplib2 ModuleNotFoundError: No module named 'httplib2'

rlfree commented 1 year ago

It looks like you just need to reinstall the requirements.txt file. You can use: pip3.7 install -r requirements.txt Or python3.7 -m pip install -r requirements.txt

macgreine commented 1 year ago

Using this distro..there isn't any requirements.txt https://github.com/Conso1eCowb0y/Deepminer

rlfree commented 1 year ago

Sorry, different project. For each of these modules coming up as missing you'll have to install with pip3.7. So for httplib2 you'll use the same commands as above but change the module name.

macgreine commented 1 year ago

Installed httplib2 and now..drum roll please File "/usr/local/lib/python3.7/http/client.py", line 1272, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1032, in _send_output self.send(msg) File "/usr/local/lib/python3.7/http/client.py", line 972, in send self.connect() File "/usr/local/lib/python3.7/http/client.py", line 1447, in connect server_hostname=server_hostname) File "/usr/local/lib/python3.7/ssl.py", line 423, in wrap_socket session=session File "/usr/local/lib/python3.7/ssl.py", line 870, in _create self.do_handshake() File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1091)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "deepminer.py", line 6, in inputList = inputList() File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 155, in inputList ahmiaLinks = ahmia() File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 48, in ahmia with request.urlopen(req) as response: File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/usr/local/lib/python3.7/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/local/lib/python3.7/urllib/request.py", line 543, in _open '_open', req) File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain result = func(*args) File "/usr/local/lib/python3.7/urllib/request.py", line 1393, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/local/lib/python3.7/urllib/request.py", line 1352, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1091)> I am sure there is an easy fix but somehow it escapes us.

macgreine commented 1 year ago

It's Saturday night, nothing better to do than hold my hand through this process

rlfree commented 1 year ago

It appears this is an issue with how httplib2 is configured. I'll take a look at the code next week and see if I can find a workaround.

macgreine commented 1 year ago

Thanks, have a great weekend

rlfree commented 1 year ago

Try an older version of httplib2. Use this command:

python3 -m pip install httplib2==0.18.1

macgreine commented 1 year ago

I suppose you are somewhat perplexed, I know I am..This is like the old days with car repair...you fix one thing and another springs up. Installed httplib==0.18.1 no errors Rebooted python3.7 deepminer.py and: Traceback (most recent call last): File "/usr/local/lib/python3.7/urllib/request.py", line 1350, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/usr/local/lib/python3.7/http/client.py", line 1277, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1323, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1272, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1032, in _send_output self.send(msg) File "/usr/local/lib/python3.7/http/client.py", line 972, in send self.connect() File "/usr/local/lib/python3.7/http/client.py", line 1447, in connect server_hostname=server_hostname) File "/usr/local/lib/python3.7/ssl.py", line 423, in wrap_socket session=session File "/usr/local/lib/python3.7/ssl.py", line 870, in _create self.do_handshake() File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1091)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "deepminer.py", line 6, in inputList = inputList() File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 155, in inputList ahmiaLinks = ahmia() File "/home/kali/Apps/Deepminer/modules/deephelpers.py", line 48, in ahmia with request.urlopen(req) as response: File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/usr/local/lib/python3.7/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/local/lib/python3.7/urllib/request.py", line 543, in _open '_open', req) File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain result = func(*args) File "/usr/local/lib/python3.7/urllib/request.py", line 1393, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/local/lib/python3.7/urllib/request.py", line 1352, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1091)>

rlfree commented 1 year ago

This is likely because the version of the Python libraries used originally are now deprecated versions. I'll likely archive this project until I have some free time to port it over to a newer version of Python.

macgreine commented 1 year ago

Thanks for all your efforts...Now, I have to put learning Python on my to do list. Again you have been very helpful..above beyond the call of duty. Have a great week!