defaultUserAgent not overridden through the command line

[humblekofe]

Describe the bug:

When calling ZAP via Maven including the following zapOptions

-config network.connection.httpProxy.enabled=true -config network.connection.httpProxy.host=my.proxy.host -config network.connection.httpProxy.port=80 -config network.connection.defaultUserAgent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0"

the Connect request to the proxy server to "cfu.zaproxy.org:443" is sent out with the following useragent: "Mozilla/5.0 (Windows NT 10.0; rv:125.0) Gecko/20100101 Firefox/125.0" regardless of what is configured above. Although zap.log confirms that the Firefox 128 Useragent has been configured, it effectively sends out the HTTP CONNECT Request to the Proxy using the Firefox 125 Useragent.

The resulting problem in my context is that the Firefox 125 Useragent is blocked by the Proxy that I'm using and I have to reconfigure it to something else using the defaultUserAgent.

Additional information: The reconfiguration of the useragent works as expected when I do it in the OWASP ZAP UI, i.e. the Proxy Connect Request carries the Useragent as specified under Tools -> Options -> Network -> Connection -> DefaultUserAgent. This is also reflected as entered in the config.xml file. As a result, checking and updating plugins works when using the ZAP-UI but not in the maven build.

Steps to reproduce the behavior:

To reproduce this using Wireshark, capture the network traffic while running a ZAP from a maven plugin br.com.softplan.security.zap:zap-maven-plugin with the following options (some of the do not matter here but I'll provide them here as I'm not 100% sure if they are relevant):

-daemon -silent -config start.checkForUpdates=false -config ajaxSpider.numberOfBrowsers=1 -config network.ratelimit.rules.rule.enabled=true -config network.connection.httpProxy.enabled=true -config network.connection.httpProxy.host=80 -config network.connection.httpProxy.port=80 -config network.connection.httpProxy.exclusions.exclusion.host=${owaspZapTargetHost} -config network.connection.defaultUserAgent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0" -config network.connection.timeoutInSecs=300 -dir ${project.build.directory}/owasp-zap-work -addoninstall pscanrulesAlpha -addoninstall pscanrulesBeta -addoninstall pscanrules -addoninstall ascanrulesAlpha -addoninstall ascanrulesBeta -addoninstall ascanrules -addoninstall spider -addoninstall spiderAjax -addoninstall reports -addoninstall webdriverwindows

In Wireshark you will see the following packet sent: DUMMY-TXjE@ /%PO6 PPbCONNECT cfu.zaproxy.org:443 HTTP/1.1 Host: cfu.zaproxy.org:443 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0

If one has the possibility to use a proxy that blocks the Firefox 125 useragent, the error output as described below in the zap.log section can be observed.

Expected behavior:

The useragent used for proxy requests using zapOptions and -config network.connection.defaultUserAgent should be used for proxy requests e.g. to cfu.zaproxy.org

Software versions:

Zap 2.15 coming with network-beta-0.16.0.zap

Screenshots:

No response

Errors from the zap.log file:

2024-11-04 18:43:48,866 [main ] INFO DaemonBootstrap - ZAP 2.15.0 started 04/11/2024, 18:43:48 with home: C:\DEV\lf-empty\master\src\lf-test-customer\ipl-customer-test-owasp-zap\target\owasp-zap-work\ cores: 20 maxMemory: 512 MB ... 2024-11-04 18:43:48,895 [main ] INFO AbstractParam - Setting config network.connection.httpProxy.enabled = true was null 2024-11-04 18:43:48,895 [main ] INFO AbstractParam - Setting config network.connection.httpProxy.host = my.proxy.server was null 2024-11-04 18:43:48,895 [main ] INFO AbstractParam - Setting config network.connection.httpProxy.port = 80 was null 2024-11-04 18:43:48,896 [main ] INFO AbstractParam - Setting config network.connection.defaultUserAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0 was null 2024-11-04 18:43:48,896 [main ] INFO AbstractParam - Setting config network.connection.timeoutInSecs = 300 was null

... 2024-11-04 18:43:52,287 [ZAP-cfu] ERROR ExtensionCallHome - CONNECT refused by proxy: HTTP/1.1 403 Forbidden org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy: HTTP/1.1 403 Forbidden

I have also configured logging in zap.log to debug, but this does not produce additional information regarding the UserAgent contained in the Request, I was only able to see this using Wireshark

Additional context:

No response

Would you like to help fix this issue?

• [ ] Yes

[kingthorin]

zap-maven-plugin is not maintained by the zaproxy team. The top two results for that from google have no been updated in 6-9 years, if you can figure out which you're using you could try to open an issue but I doubt it'll go anywhere.

[humblekofe]

Hi @kingthorin , thanks a lot for your immediate response!

I have done further investigations and would kindly ask you to take a second look at this new information as I have the impression that the root cause for my problem is not the maven-plugin itself because it only generates a commandline which it uses to invoke Zap.

When I call ZAP using the following java command:

java -Xmx512m -Dhttps.proxyHost=my.proxy.host -Dhttps.proxyPort=80 -Dhttp.proxyHost=my.proxy.host -Dhttp.proxyPort=80 -Dhttp.agent= -jar zap-2.15.0.jar -daemon -silent -dir C:\Temp\ZAP\OWASP-ZAP\temp -config start.checkForUpdates=false -config ajaxSpider.numberOfBrowsers=1 -config api.disablekey=true -config api.incerrordetails=true -config network.localServers.mainProxy.enabled=false -config network.connection.httpProxy.enabled=true -config network.connection.httpProxy.host=my.proxy.host -config network.connection.httpProxy.port=80 **-config network.connection.defaultUserAgent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0"** -config network.connection.timeoutInSecs=300 -dir C:\Temp\ZAP -addoninstall pscanrulesAlpha -addoninstall pscanrulesBeta -addoninstall pscanrules -addoninstall ascanrulesAlpha -addoninstall ascanrulesBeta -addoninstall ascanrules -addoninstall spider -addoninstall spiderAjax -addoninstall reports -addoninstall webdriverwindows -port 8290

and inspect the useragent using wireshark, all requests to the proxy still contain the Firefox 125 Useragent instead of the Firefox 128 as given via the commandline.

I might be wrong but my understanding is that the reason is not the zap-maven-plugin (which is basically just a wrapper for the java invocation) but the Owasp zap itself...

Thanks a lot for your help! Felix

[thc202]

Can you provide the zap.log? I'm interested to know if the network add-on is being updated.

[humblekofe]

No it isn't, none of the plugins is updated as the request to cfu.zaproxy.org:443 fails which is used to retrieve the latest plugin versions (at least this is my understanding). Actually none of the plugins that would be updated or installed is downloaded or installed.

The rows reflecting the update ...Downloading add-on ... network until ...Finished installing new addon network v0.18.0 do not appear and finally org.parosproxy.paros.CommandLine - Check for updates call failed is logged.

Zap 2.15 comes with network-beta-0.16.0.zap and while network-beta-0.18.0.zap is currently the latest version. I have also updated it in the bundle by placing the network-beta-0.18.0.zap manually in the plugin directory to see it this changes anything but the behaviour was the same.

[humblekofe]

May I ask if you have considered reopening this issue based on the addtional information above?

[humblekofe]

@thc202 or @kingthorin : Just in case my last comment went by unseen: Is my understanding correct that my test described on 5-NOV above potentially describes a problem within Owasp Zap? If that's the case could you please reopen this ticket? Thanks for your help!

[thc202]

It was not, I just didn't confirm yet the issue but happy to reopen.