search

xmendez / wfuzz

Web application fuzzer
http://wfuzz.io
GNU General Public License v2.0
5.96k stars 1.39k forks source link

Scan Fails with pycurl.error: cannot invoke setopt() - no curl handle #120

Open ossie-git opened 5 years ago

ossie-git commented 5 years ago

Issue template

Context

Please check:

Please describe your local environment:

Wfuzz version: Output of wfuzz --version

2.3.4

Python version: Output of python --version

Python 2.7.15rc1

Python 3 is also installed: Python 3.6.7

OS: Ubuntu 18.04.1 LTS

Report

What is the current behavior?

wfuzz exits without completing the scan and throws a: "pycurl.error: cannot invoke setopt() - no curl handle" error. A similar issue is found here - https://github.com/xmendez/wfuzz/issues/75 - but that ticket refers to accessing it programmatically. This behavior is repeated when I repeat the scan with the same parameters

I run the following scan:

wfuzz -c --hc 404,500 -w wordlist.txt https://www.example.com/FUZZ

After some scanning, it throws the following error:

000371:  C=200     31 L       55 W          701 Ch        "."
003010:  C=500     44 L      213 W         2058 Ch        "albums.php"
Fatal exception: Pycurl error 35: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.example.com:443
Exception in thread _read_retry_queue:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.6/dist-packages/wfuzz/myhttp.py", line 179, in _read_retry_queue
    self.enqueue(res, poolid)
  File "/usr/local/lib/python3.6/dist-packages/wfuzz/myhttp.py", line 87, in enqueue
    c = fuzzres.history.to_http_object(self.freelist.get())
  File "/usr/local/lib/python3.6/dist-packages/wfuzz/fuzzobjects.py", line 399, in to_http_object
    return Request.to_pycurl_object(c, self._request)
  File "/usr/local/lib/python3.6/dist-packages/wfuzz/externals/reqresp/Request.py", line 287, in to_pycurl_object
    c.setopt(pycurl.MAXREDIRS, 5)
pycurl.error: cannot invoke setopt() - no curl handle

What is the expected or desired behavior?

The scan should complete successfully

Please provide steps to reproduce, including exact wfuzz command executed and output:

Refer to the above

Other relevant information:

X

xmendez commented 5 years ago

Hi,

Any chances that you can share the URL failing? it seems a problem with SSL.

Cheers

naripok commented 4 years ago

Hi guys, I'm trying to fuzz the Postman box, from 'hackthebox', and I'm getting a similar issue.

uname -all
linux-kali-5.3.0-kali3-amd64 #1 SMP Debian-5.3.15-1kali1 (2019-12-09) x86_64 GNU/Linux

using the command with kali's install of wfuzz:

python -V  
Python 2.7.17
wfuzz --version 
warning: Pycurl was not compiled against OpenSSL. Wfuzz may not work correctly when fuzzing SSL sites. Check Wfuzz's documentation for more information.
2.4

wfuzz -c -w ~/SecLists/Passwords/(pick a list) https://10.10.10.160/FUZZ

the scan fail to start and return the error: Fatal exception: Pycurl error 56: GnuTLS recv error (-110): The TLS connection was non-properly terminated and spits out the same stack trace as OP.

So I tried to compile pycurl against OpenSSL ans give it a try, also using the last version of wfuzz:

python -V  
Python 3.7.5
wfuzz --version 
2.4.2

wfuzz -c -w ~/SecLists/Passwords/(pick a list) https://10.10.10.160/FUZZ

the scan fail to start and return the error: Fatal exception: Pycurl error 56: Open_SSL SSL_read: Success

and spits out the same stack trace as OP also.

If I try to use it for non https sites, it works flawlessly. Also, if I try to fuzz google by https, it also works, so it is not a SSL thing nether.

There is one importante thing, I believe, I noticed: When I try to visit the site using the browser (firefox), it gives me the error: MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT

I'm willing to help to debug this issue, so if there is anything I can do to help, let me know. I love your tool and dont wanna live without it lool.

Thx fellow.

Edit: Forgot to mention: If I add the -Z flag to the scan, wfuzz will continue to scan using the word list, but failing and throwing the same error Fatal exception: Pycurl error 56: Open_SSL SSL_read: Success on every line.

Update: I managed to get the pycurl verbose information about the requests, so there is an example:

* Hostname 10.10.10.160 was found in DNS cache
*   Trying 10.10.10.160:10000...
* TCP_NODELAY set
* Connected to 10.10.10.160 (10.10.10.160) port 10000 (#5)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSL re-using session ID
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: O=Webmin Webserver on Postman; CN=*; emailAddress=root@Postman
*  start date: Aug 25 16:26:22 2019 GMT
*  expire date: Aug 23 16:26:22 2024 GMT
*  issuer: O=Webmin Webserver on Postman; CN=*; emailAddress=root@Postman
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST / HTTP/1.1
Host: 10.10.10.160:10000
Accept: */*
Content-Type: application/x-www-form-urlencoded
User-Agent: Wfuzz/2.4
Content-Length: 25

* upload completely sent off: 25 out of 25 bytes
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 200 Document follows
< Date: Thu, 26 Dec 2019 13:46:01 GMT
< Server: MiniServ/1.910
< Connection: close
< Auth-type: auth-required=1
< Set-Cookie: redirect=1; path=/
< Set-Cookie: testing=1; path=/; secure
< X-Frame-Options: SAMEORIGIN
< Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval'; frame-src 'self'; child-src 'self'
< Content-type: text/html; Charset=UTF-8
<
* OpenSSL SSL_read: Success
* Closing connection 5

I think pycurl is failing to identify the received data and close the connection or something. Also, the stack trace mentioned seems to be caused by starvation of connections on the free/retry queues. If you add an if statement ignoring the errno 56 on myhttp.py, line 235, the scan will run indefinitely:

myhttp.py

235    if errno not in [56]:
236        e = FuzzExceptNetError("Pycurl error %d: %s" % (errno, errmsg))
237        self.pool_map[poolid]["queue"].put(res.update(exception=e))

There is another piece of information. Searching on pycurl docs, I found this ref http://pycurl.io/docs/latest/troubleshooting.html#pycurl-error-1 that points at the curl header definitions, and at line 551 you will find:

551   CURLE_RECV_ERROR,              /* 56 - failure in receiving network data */

So, it points me to think that it is indeed the case of a failure in identify received data and terminate the request.

From stackoverflow: https://stackoverflow.com/questions/17437950/when-does-an-http-1-0-server-close-the-connection

In HTTP 1.0, the server always closes the connection after sending the response UNLESS the client sent a Connection: keep-alive request header and the server sent a Connection: keep-alive response header. If no such response header exists, the client must close its end of the connection after receiving the response.

I'm not well-versed at the HTTP protocol, but maybe it is the case that wfuzz is failing to identify and close the connection? When trying to do the same request with curl, it succeeds.