w3af freezes

[andresriancho]

The problem

GET http://194.132.x.x/dvwa/?view=http://194.132.x.x.w3af.org/ returned HTTP code "200" (id=2905,from_cache=0,grep=1,rtt=3.51)
GET http://194.132.x.x/dvwa/?view=//194.132.x.x.w3af.org returned HTTP code "200" (id=2906,from_cache=0,grep=1,rtt=3.32)
GET http://194.132.x.x/dvwa/test.php?mode=97' AND '97'='98 returned HTTP code "200" (id=2907,from_cache=0,grep=1,rtt=0.53)

<more than 3 minutes here without a log message>

Failed to HTTP "POST" "http://194.132.x.x/dvwa/vulnerabilities/exec/". Reason: "HTTP timeout error", going to retry.
ExtendedUrllib error rate is at 3%
Re-sending request "<HTTPRequest "http://194.132.x.x/dvwa/vulnerabilities/exec/" (cookies:True, cache:False)>" after initial exception: "HTTP timeout error"
console.flush() took 0.00 seconds to run
text_file.flush() took 0.00 seconds to run

I would have expected for a timeout error to block one of the threads which sends HTTP requests, not all of them. Also, if the threads which send HTTP requests are locked or something, I would still expect these lines appearing every ~30 seconds:

[Mon Nov 27 14:26:40 2017 - debug] console.flush() took 0.00 seconds to run
[Mon Nov 27 14:26:40 2017 - debug] text_file.flush() took 0.00 seconds to run
[Mon Nov 27 14:26:42 2017 - debug] xml_file.flush() took 2.87 seconds to run

It seems that this is not an output plugin / manager issue, since when the output is frozen the framework is not sending HTTP requests.

Debugging

• Disabled the grep plugins by changing grep consumer's should_grep to always return False and the freezing stopped.
• Re-enabled the feature and the freezes came back (even with a nice perf improvement I added)
• By running different scans with different disabled code sections I'm trying to understand if the problem is in the writing of the messages to the CachedQueue, the reading, or the processing of those messages.
• The problem doesn't seem to be in the process of writing to CachedQueue (from grep consumer)

Lines that seem to generate lock

I just replaced the call to grep_wrapper with a sleep:

            #try:
            #    plugin.grep_wrapper(request, response)
            #except Exception, e:
            #    self.handle_exception('grep', plugin.get_name(), request, e)
            time.sleep(0.3)

And the scan finished. So it seems that one or more of the enabled grep plugins is freezing the scan by adding a lock.

[andresriancho]

Works:

#grep symfony
#grep get_emails
#grep dom_xss
#grep path_disclosure
#grep cross_domain_js
#grep strange_http_codes
#grep click_jacking
#grep strange_parameters
#grep ssn
#grep credit_cards
#grep directory_indexing
#grep http_auth_detect
#grep strange_headers
#grep code_disclosure
grep analyze_cookies

[andresriancho]

Freezes (a little bit, not much)

#grep symfony
#grep get_emails
#grep dom_xss
#grep path_disclosure
#grep cross_domain_js
#grep strange_http_codes
#grep click_jacking
#grep strange_parameters
#grep ssn
#grep credit_cards
#grep directory_indexing
#grep http_auth_detect
#grep strange_headers
grep code_disclosure
grep analyze_cookies

[andresriancho]

Just noticed that these two are also enabled by plugin dependencies:

error_pages
error_500

[andresriancho]

Freezes ~10 seconds

#grep symfony
#grep get_emails
#grep dom_xss
#grep path_disclosure
#grep cross_domain_js
#grep strange_http_codes
#grep click_jacking
#grep strange_parameters
#grep ssn
#grep credit_cards
#grep directory_indexing
#grep http_auth_detect
grep strange_headers
grep code_disclosure
grep analyze_cookies

[andresriancho]

Freezes ~15 seconds

#grep symfony
#grep get_emails
#grep dom_xss
#grep path_disclosure
#grep cross_domain_js
#grep strange_http_codes
grep click_jacking
grep strange_parameters
grep ssn
grep credit_cards
grep directory_indexing
grep http_auth_detect
grep strange_headers
grep code_disclosure
grep analyze_cookies

[andresriancho]

This freezes for at least 60 seconds:

grep symfony
grep get_emails
grep dom_xss
grep path_disclosure
grep cross_domain_js
grep strange_http_codes
#grep click_jacking
#grep strange_parameters
#grep ssn
#grep credit_cards
#grep directory_indexing
#grep http_auth_detect
#grep strange_headers
#grep code_disclosure
#grep analyze_cookies

[andresriancho]

Also freezes for at least 60 seconds:

grep symfony
grep get_emails
grep dom_xss
grep path_disclosure
#grep cross_domain_js
#grep strange_http_codes
#grep click_jacking
#grep strange_parameters
#grep ssn
#grep credit_cards
#grep directory_indexing
#grep http_auth_detect
#grep strange_headers
#grep code_disclosure
#grep analyze_cookies

[andresriancho]

No lock with:

grep symfony
grep get_emails
#grep dom_xss
#grep path_disclosure
#grep cross_domain_js
#grep strange_http_codes
#grep click_jacking
#grep strange_parameters
#grep ssn
#grep credit_cards
#grep directory_indexing
#grep http_auth_detect
#grep strange_headers
#grep code_disclosure
#grep analyze_cookies

So the problem seems to be with:

grep dom_xss
grep path_disclosure

Or

grep symfony
grep get_emails
grep dom_xss
grep path_disclosure

[andresriancho]

Long freeze with:

grep dom_xss
grep path_disclosure

[andresriancho]

Got a long freeze only when enabling:

grep path_disclosure

[andresriancho]

Theory: The grep plugin is locking the KB or DBMS, other parts of the framework want to read from it and since it is locked that is impossible.

[andresriancho]

The bug seems to be in the esm.c external lib

[andresriancho]

Fixed by https://github.com/andresriancho/w3af/tree/feature/acora