opsdisk / metagoofil

Search Google and download specific file types
Other
415 stars 98 forks source link

python3.10 Running Error #35

Closed GGStudy-DDUp closed 2 months ago

GGStudy-DDUp commented 10 months ago

I encountered this problem after running.

$ proxychains -q python3 metagoofil.py -d nsfocus.com -f -t all -e 60 
[*] Searching for 100 .all files and waiting 60.0 seconds between searches
[-] EXCEPTION: <urlopen error EOF occurred in violation of protocol (_ssl.c:997)>
Traceback (most recent call last):
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1454, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/usr/local/python3/lib/python3.10/ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/python3/lib/python3.10/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/usr/local/python3/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/osint/osint/domain_col/tool/metagoofil/metagoofil.py", line 162, in go
    for url in googlesearch.search(
  File "/usr/local/python3/lib/python3.10/site-packages/googlesearch/__init__.py", line 272, in search
    get_page(url_home % vars(), user_agent, verify_ssl)
  File "/usr/local/python3/lib/python3.10/site-packages/googlesearch/__init__.py", line 174, in get_page
    response = urlopen(request)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:997)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/osint/osint/domain_col/tool/metagoofil/metagoofil.py", line 381, in <module>
    mg.go()
  File "/home/osint/osint/domain_col/tool/metagoofil/metagoofil.py", line 175, in go
    if e.code == 429:
AttributeError: 'URLError' object has no attribute 'code'
opsdisk commented 10 months ago

Hi @GGStudy-DDUp - is it possible for you to run it without proxychains? I'd like to rule that out first.

Another note, you must specify the case-sensitive -t ALL to search for all file extensions, otherwise it will search for just the .all file extension.

GGStudy-DDUp commented 10 months ago

Hi @GGStudy-DDUp - is it possible for you to run it without proxychains? I'd like to rule that out first.

Another note, you must specify the case-sensitive -t ALL to search for all file extensions, otherwise it will search for just the .all file extension.

I have configured the proxy and it works.

round_robin_chain
chain_len = 1
proxy_dns
remote_dns_subnet 224
tcp_read_time_out 15000
tcp_connect_time_out 8000
[ProxyList]
socks5  192.168.190.1 1080
socks5  192.168.190.1 7890

image

And I also used -t ALL to run it, but this error occurred.

opsdisk commented 10 months ago

Can you run the same metagoofil command without a proxy / proxychains?

python3 metagoofil.py -d nsfocus.com -f -t all -e 60

At first glance, this appears to be a proxy / proxychains configuration issue, and not a metagoofil one. Someday, I'd like to rewrite metagoofil with this library (https://github.com/opsdisk/yagooglesearch/) which has better proxy support.

For proxychains troubleshooting, try:

GGStudy-DDUp commented 10 months ago

Can you run the same metagoofil command without a proxy / proxychains?

python3 metagoofil.py -d nsfocus.com -f -t all -e 60

At first glance, this appears to be a proxy / proxychains configuration issue, and not a metagoofil one. Someday, I'd like to rewrite metagoofil with this library (https://github.com/opsdisk/yagooglesearch/) which has better proxy support.

For proxychains troubleshooting, try:

  • Commenting out this line socks5 192.168.190.1 7890
  • Check if there is a /etc/proxychains4.conf file as well
  • Specifying the configuration file you have: proxychains -f path/to/proxychains/config/file

I tried it and it got the error running in my virtual machine. But it worked fine on my VPS machine.

opsdisk commented 10 months ago

got the error running in my virtual machine

Is that with or without proxychains and the proxy?

GGStudy-DDUp commented 10 months ago

got the error running in my virtual machine

Is that with or without proxychains and the proxy?

Maybe I have some environmental problems, but my agent is normal. Thank you for your help.

opsdisk commented 10 months ago

Let me know if you want to keep this issue open @GGStudy-DDUp

opsdisk commented 2 months ago

Bump @GGStudy-DDUp