search

zaproxy / browser-extension

Apache License 2.0
23 stars 7 forks source link

client: headless Chrome high CPU usage using AJAX Spider #81

Closed double16 closed 10 months ago

double16 commented 11 months ago

Describe the bug

The client add-on can cause high CPU usage in headless Chrome, the spider does not progress and client information is not sent to ZAP.

Steps to reproduce the behavior

Please note this example is for a public site with a bug bounty program. In the future the program may not be in effect.

Desktop:

  1. Install 'client' and 'selenium' add-ons
  2. Create a new context
  3. Ensure "In Scope" is ticked
  4. Add to "Include in Context" "https://www.humblebundle.com.*"
  5. Save the context
  6. Click Tools > AJAX Spider
  7. Set the starting URL to "https://www.humblebundle.com"
  8. Set browser to "Chrome Headless"
  9. Set the number of concurrent browsers to "1"
  10. Open "Client History" panel
  11. Run the spider and within 3-5 minutes the spider will not continue and the chrome process will have high CPU usage, spider will not progress, client history stops receiving data.

Automation:

# example demonstrating client side integration causing high CPU usage in chrome headless
# ZAP Weekly 2023-10-09
# Add-ons:
#   ascanrulesAlpha-alpha-44.zap
#   attacksurfacedetector-alpha-1.1.4.zap
#   authstats-alpha-2.zap
#   callgraph-alpha-5.zap
#   client-0.3.0.zap (git commit hash b624ca0a065c016234a01ff7f9b214247ff2bc19)
#   communityScripts-alpha-17.zap
#   custompayloads-alpha-0.12.0.zap
#   directorylistv2_3-release-4.zap
#   evalvillain-alpha-0.3.0.zap
#   fileupload-alpha-1.1.0.zap
#   fuzzdb-release-9.zap
#   fuzzdboffensive-release-4.zap
#   highlighter-alpha-8.zap
#   imagelocationscanner-beta-4.zap
#   jsonview-alpha-3.zap
#   jwt-alpha-1.0.3.zap
#   jython-beta-13.zap
#   levoai-zap-addon-alpha-0.2.0.zap
#   neonmarker-alpha-1.6.0.zap
#   network-beta-0.11.2.zap
#   packpentester-alpha-0.1.0.zap
#   paramdigger-alpha-0.2.0.zap
#   postman-alpha-0.1.0.zap
#   pscanrulesAlpha-alpha-41.zap
#   quickstart-release-42.zap
#   reflect-alpha-0.0.11.zap
#   revisit-alpha-4.zap
#   saml-alpha-10.zap
#   selenium-release-15.15.0.zap
#   sqliplugin-beta-15.zap
#   sse-alpha-12.zap
#   svndigger-release-4.zap
#   tokengen-beta-15.zap
#   treetools-beta-8.zap
#   viewstate-alpha-3.zap
#   wappalyzer-release-21.24.0.zap
#   webdriverlinux-release-63.zap
#   webdrivermacos-release-63.zap
#   webdriverwindows-release-63.zap
#
#   OS:
#     Distributor ID: Kali
#     Description:    Kali GNU/Linux Rolling
#     Release:        2023.3
#     Codename:       kali-rolling
#     Linux kali 6.3.0-kali1-amd64 zaproxy/zaproxy#1 SMP PREEMPT_DYNAMIC Debian 6.3.7-1kali1 (2023-06-29) x86_64 GNU/Linux

env:
  contexts :
    - name: context 1
      urls:
        - https://www.humblebundle.com/
      includePaths:
        - https://www.humblebundle.com.*
      sessionManagement:
        method: cookie
  parameters:
    failOnError: true
    failOnWarning: false
    progressToStdout: true

jobs:

  - type: passiveScan-config
    name: passiveScan-config
    parameters:
      maxAlertsPerRule: 0
      maxBodySizeInBytesToScan: 0
      scanOnlyInScope: false

  - type: spiderAjax
    name: spiderAjax
    parameters:
      browserId: chrome-headless
      clickDefaultElems: true
      clickElemsOnce: true
      context:
      eventWait: 1000
      maxCrawlDepth: 10
      maxCrawlStates: 0
      maxDuration: 60
      numberOfBrowsers: 1
      randomInputs: true
      reloadWait: 1000
      runOnlyIfModern: false
      url: https://www.humblebundle.com/
      user:

  - type: delay
    name: delay
    parameters:

  - type: passiveScan-wait
    name: passiveScan-wait
    parameters:
      maxDuration: 0

  - type: report
    name: report
    parameters:
      displayReport: false
      reportDescription:
      reportDir: /home/vagrant
      reportFile: null
      reportTitle: ZAP Scanning Report
      template: risk-confidence-html
      theme: null

Expected behavior

  1. AJAX spider progresses as without the client add-on installed.
  2. The chrome process uses CPU usage consistent without client add-on installed.
  3. Client information is received per plugin intent.

Software versions

# ZAP Weekly 2023-10-09
# Add-ons:
#   ascanrulesAlpha-alpha-44.zap
#   attacksurfacedetector-alpha-1.1.4.zap
#   authstats-alpha-2.zap
#   callgraph-alpha-5.zap
#   client-0.3.0.zap (git commit hash b624ca0a065c016234a01ff7f9b214247ff2bc19)
#   communityScripts-alpha-17.zap
#   custompayloads-alpha-0.12.0.zap
#   directorylistv2_3-release-4.zap
#   evalvillain-alpha-0.3.0.zap
#   fileupload-alpha-1.1.0.zap
#   fuzzdb-release-9.zap
#   fuzzdboffensive-release-4.zap
#   highlighter-alpha-8.zap
#   imagelocationscanner-beta-4.zap
#   jsonview-alpha-3.zap
#   jwt-alpha-1.0.3.zap
#   jython-beta-13.zap
#   levoai-zap-addon-alpha-0.2.0.zap
#   neonmarker-alpha-1.6.0.zap
#   network-beta-0.11.2.zap
#   packpentester-alpha-0.1.0.zap
#   paramdigger-alpha-0.2.0.zap
#   postman-alpha-0.1.0.zap
#   pscanrulesAlpha-alpha-41.zap
#   quickstart-release-42.zap
#   reflect-alpha-0.0.11.zap
#   revisit-alpha-4.zap
#   saml-alpha-10.zap
#   selenium-release-15.15.0.zap
#   sqliplugin-beta-15.zap
#   sse-alpha-12.zap
#   svndigger-release-4.zap
#   tokengen-beta-15.zap
#   treetools-beta-8.zap
#   viewstate-alpha-3.zap
#   wappalyzer-release-21.24.0.zap
#   webdriverlinux-release-63.zap
#   webdrivermacos-release-63.zap
#   webdriverwindows-release-63.zap
#
#   OS:
#     Distributor ID: Kali
#     Description:    Kali GNU/Linux Rolling
#     Release:        2023.3
#     Codename:       kali-rolling
#     Linux kali 6.3.0-kali1-amd64 zaproxy/zaproxy#1 SMP PREEMPT_DYNAMIC Debian 6.3.7-1kali1 (2023-06-29) x86_64 GNU/Linux

Screenshots

No response

Errors from the zap.log file

No response

Additional context

No response

Would you like to help fix this issue?

kocoten1992 commented 11 months ago

I second this, when I use ZAP, I notice sometime CPU go as high as 100% on all cpu core (16 cores), especially when the page is cloudflare tickbox verification (image copy from the internet - but it was this tickbox page)

image

thc202 commented 11 months ago

Do you have the client add-on installed?

kocoten1992 commented 11 months ago

Do you have the client add-on installed?

No, it was the default installation, but now I figure what I was talking about is different from this issue (this issue is about spider, but I was using chrome Manual Explore).

psiinon commented 10 months ago

Just open https://www.humblebundle.com/ in Firfox - with extra logging I can see that we just get a never ending stream of events. Thats not right - looking into that...

psiinon commented 10 months ago

For info I've just run the AJAX SPider against https://www.humblebundle.com/ with the latest version of the ZAP extension and it ran to completion! It also found a lot of URLs that the AJAX Spider "missed"

psiinon commented 10 months ago

The new version is added in https://github.com/zaproxy/zap-extensions/pull/5027