search

zaproxy / zap-hud

The ZAP Heads Up Display (HUD)
Apache License 2.0
253 stars 151 forks source link

daemon mode always crashes #474

Closed nothingismagick closed 5 years ago

nothingismagick commented 5 years ago

I think that this may be due to something wrong with Firefox - but the error output is a little mysterious. If I don't use -daemon then it works (but I have to click through the GUI manually, and I MUST use chrome).

Is there a secret command for choosing to launch a different browser? using -browser chrome makes it hang... 

$ bash ./zap/ZAP_D-2019-04-29/zap.sh -daemon -host 127.0.0.1 -port 8082  -hud -nostdout
Found Java version 1.8.0_181
Available memory: 16384 MB
Using JVM args: -Xmx4096m
ZAP is now listening on 127.0.0.1:8082
1556662771252   geckodriver     INFO    geckodriver 0.19.1
1556662771260   geckodriver     INFO    Listening on 127.0.0.1:37133
1556662771570   mozrunner::runner       INFO    Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-profile" "/var/folders/3r/kb0m2qy90fdb338ydcrpvw8r0000gn/T/rust_mozprofile.uI7qyhyYofFR"
1556662772093   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1556662772093   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1556662772093   addons.webextension.screenshots@mozilla.org     WARN    Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1556662772238   Marionette      TRACE   Received observer notification profile-after-change
1556662772362   Marionette      TRACE   Received observer notification command-line-startup
1556662772362   Marionette      TRACE   Received observer notification nsPref:changed
1556662772363   Marionette      DEBUG   Init aborted (running=false, enabled=true, finalUIStartup=false)
1556662772605   Marionette      TRACE   Received observer notification toplevel-window-ready
1556662773396   Marionette      TRACE   Received observer notification sessionstore-windows-restored
1556662773396   Marionette      TRACE   Waiting until startup recorder finished recording startup scripts...
1556662773696   Marionette      TRACE   All scripts recorded.
1556662773696   Marionette      DEBUG   Setting recommended pref app.update.disabledForTesting to true
1556662773696   Marionette      DEBUG   Setting recommended pref browser.contentblocking.introCount to 99
1556662773697   Marionette      DEBUG   Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1556662773697   Marionette      DEBUG   Setting recommended pref dom.disable_beforeunload to true
1556662773698   Marionette      DEBUG   Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1556662773756   Marionette      INFO    Listening on port 63764
1556662773757   Marionette      DEBUG   Remote service is active
1556662773844   Marionette      DEBUG   Accepted connection 0 from 127.0.0.1:63813
1556662773846   Marionette      DEBUG   0 -> [0,1,"newSession",{"acceptInsecureCerts":true,"browserName":"firefox","capabilities":{"desiredCapabilities":{"acceptInsecureCerts":true,"browserName":"firefox"}}}]
1556662773846   Marionette      DEBUG   0 <- [1,1,{"error":"unknown command","message":"newSession","stacktrace":"WebDriverError@chrome://marionette/content/error.js:179: ... et@chrome://marionette/content/server.js:235:8\n_onJSONObjectReady/<@chrome://marionette/content/transport.js:493:9\n"},null]
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
May 01, 2019 12:19:33 AM org.openqa.selenium.remote.ErrorCodes toStatus
INFO: HTTP Status: '404' -> incorrect JSON status mapping for 'unknown error' (500 expected)
thc202 commented 5 years ago

Which Firefox version are you using?

1556662771252 geckodriver INFO geckodriver 0.19.1

That's quite old geckodriver, ZAP bundles (and should be using) 0.24.0.

psiinon commented 5 years ago

FYI there isnt currently an option to specify Chrome, but we could add one...

nothingismagick commented 5 years ago

Firefox 66.0.3 (on MacOS) I am also using the freshest daily / weekly version of zaproxy from here:

https://raw.githubusercontent.com/zaproxy/zap-admin/master/ZapVersions.xml

nothingismagick commented 5 years ago

Allowing the user of the app-extension to choose chrome would be excellent (and important) because many Quasar devs use chrome first (and test compliance on firefox, safari etc.) Its not my preference, just something that has organically happened.

If you are thinking about this, I'd like to reference our call from last week and figure out if there is a way to collaborate with cypress and use their chrome or electron browser as a target as well.

psiinon commented 5 years ago

I've no problem with us supporting other browsers, but that will almost certainly require changes to the selenium add-on (which we use for launching the browsers). I think we'll need to know the path to the browser executable and to the related webdriver. @thc202 is that everything we would need (in addition to the selenium code changes)?

thc202 commented 5 years ago

@nothingismagick the geckodriver being used is too old for that Firefox version: https://firefox-source-docs.mozilla.org/testing/geckodriver/geckodriver/Support.html Do you know from where that geckodriver is being added? Is that on the path?

@psiinon yes, the path to the browser binary should be enough in this case (both use ChromeDriver).

nothingismagick commented 5 years ago

This might be coming from a global install of wdio (which probably has outdated module deps), but I am on mobile for the rest of the day and can't check at the moment.

nothingismagick commented 5 years ago

So, I can't find geckodriver in my path. To whit I just installed it just now like:

$ brew install geckodriver

$ geckodriver -V
geckodriver 0.24.0

$ geckodriver -v
1556735093590   webdriver::httpapi      DEBUG   Creating routes
1556735093599   geckodriver     DEBUG   Listening on 127.0.0.1:4444

So, even in that console, I still get an outdated geckodriver when running the command as I showed above. I also can confirm that wdio and webdriver are not in my path either. So it seems that the geckodriver being used is coming from zaproxy.

kingthorin commented 5 years ago

You're using a weekly so you can check addon versions from the CLI (-suppinfo). https://github.com/zaproxy/zap-core-help/wiki/HelpCmdline

nothingismagick commented 5 years ago 
$ bash ./zap/ZAP_D-2019-04-29/zap.sh -suppinfo

Found Java version 1.8.0_181 Available memory: 16384 MB Using JVM args: -Xmx4096m OWASP ZAP Version: D-2019-04-29 Installed Add-ons: [[id=accessControl, version=6.0.0], [id=alertFilters, version=8.0.0], [id=ascanrules, version=33.0.0], [id=ascanrulesBeta, version=25.0.0], [id=bruteforce, version=8.0.0], [id=coreLang, version=14.0.0], [id=diff, version=9.0.0], [id=directorylistv1, version=4.0.0], [id=formhandler, version=3.0.0], [id=fuzz, version=11.0.0], [id=gettingStarted, version=10.0.0], [id=help, version=9.0.0], [id=hud, version=0.4.0], [id=importurls, version=6.0.0], [id=invoke, version=10.0.0], [id=jxbrowser, version=14.0.0], [id=jxbrowserlinux64, version=12.0.0], [id=jxbrowsermacos, version=12.0.0], [id=jxbrowserwindows, version=12.0.0], [id=jxbrowserwindows64, version=5.0.0], [id=onlineMenu, version=7.0.0], [id=openapi, version=13.0.0], [id=plugnhack, version=12.0.0], [id=portscan, version=9.0.0], [id=pscanrules, version=24.0.0], [id=pscanrulesBeta, version=19.0.0], [id=quickstart, version=26.0.0], [id=replacer, version=8.0.0], [id=reveal, version=3.0.0], [id=saverawmessage, version=5.0.0], [id=savexmlmessage, version=0.1.0], [id=scripts, version=25.0.0], [id=selenium, version=15.0.0], [id=sequence, version=6.0.0], [id=spiderAjax, version=23.0.0], [id=tips, version=6.0.0], [id=webdriverlinux, version=9.0.0], [id=webdrivermacos, version=9.0.0], [id=webdriverwindows, version=9.0.0], [id=websocket, version=19.0.0], [id=zest, version=29.0.0]] Operating System: Mac OS X Java Version: Oracle Corporation 1.8.0_181 System's Locale: en_FR Display Locale: en_GB Format Locale: en_GB ZAP Home Directory: /redacted/Library/Application Support/ZAP_D/ ZAP Installation Directory: /redacted/test2/zap/ZAP_D-2019-04-29/./ Look and Feel: Mac OS X (com.apple.laf.AquaLookAndFeel)

thc202 commented 5 years ago

What's the path in Options > Selenium > geckodriver? If you don't have UI, you can check in the config.xml file, element selenium/firefoxDriver.

nothingismagick commented 5 years ago 
    <selenium version="2">
        <chromeDriver>/redacted/Library/Application Support/ZAP/webdriver/macos/64/chromedriver</chromeDriver>
        <firefoxDriver>/redacted/Library/Application Support/ZAP/webdriver/macos/64/geckodriver</firefoxDriver>
    </selenium>
nothingismagick commented 5 years ago

So I can confirm:

64 $ rm geckodriver && ln -s /usr/local/bin/geckodriver .

Enables this on my machine.

thc202 commented 5 years ago

It would be great to know what version that binary had though.

thc202 commented 5 years ago

OK, I see, that was picking the geckodriver from stable ZAP home dir (which is indeed older) not dev.

nothingismagick commented 5 years ago

That must be leftover from the MacOS install that I performed last week. After purging ZAP and ZAP_D everything works as expected.

nothingismagick commented 5 years ago

So instead of creating another issue (which i can do if you prefer) - I am still having problems with the harness. Now I can get firefox to open (yay) but it will not immediately go to the route I want:

$ bash ./zap/ZAP_D-2019-04-29/zap.sh -daemon -host 127.0.0.1 -port 8008 \
                      -hud -nostdout -quickurl http://localhost:8080

The quickurl is not propegated. If I then manually visit localhost:8080, the HUD will not show up.

If I remove the quickurl flag, I can then manually enter the address - and the HUD will appear.

kingthorin commented 5 years ago

quickurl is a scan flag it has nothing to do with your browser. (Do it in UI mode and you'll see what it does. Or review this: https://github.com/zaproxy/zap-core-help/wiki/HelpAddonsQuickstartCmdline)

nothingismagick commented 5 years ago

Thankyou! So is there a way for me to tell the HUD where to go right away?

nothingismagick commented 5 years ago

Because it looks like i should be able to place that here:

1556805773506   mozrunner::runner       INFO    Running command:  \ 
"/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" \
 "-no-remote" "-profile"

aka "-new-window http://localhost:8080" or something along those lines.

kingthorin commented 5 years ago

Thankyou! So is there a way for me to tell the HUD where to go right away?

Not that I'm aware of.

nothingismagick commented 5 years ago

I'll close this issue because my problem was solved by symlinking the up to date geckodriver into the manually installed macOS version of zap - or removing the manually installed version entirely.