--proxy global support

[Regala]

Hey! Wondering if the --proxy option could be supported globally. This would assist tremendously when testing in conjunction with Burp.

Thank you 💚

[minight]

I think this would be a good feature to add. It may increase the allocations in our result hotloop, however it may be something that the user acknowledges it may slow down the application.

@Regala, a few questions for the usecase you intend for:

• do you only want (non-wildcard) results to go through the proxy? or all requests
• would you still use the feature if it caused scanning to run at 50% speed. Is the feature a higher priority than the speed of the scanning?

For implementation, a simple implementation might be a post-request hook like the current PrintResults is.

There are a few concerns prior to implementing this that need to be ironed out:

• Our current proxy implementation was hacked together with little concern for performance since it is a singleshot usecase. We additionally have some variation between the requests performed by the engine and by the replay function due to using net/http instead of fasthttp. Can we use fasthttp to perform a proxied request?
  • Potentially yes https://pkg.go.dev/github.com/valyala/fasthttp/fasthttpproxy?utm_source=godoc with https://github.com/valyala/fasthttp/issues/363
• How to we avoid blocking our main request loop with proxy requests, or is this a tradeoff we willingly make given its already a performance penalty to run this.
  • just implementing it as another post-request hook in our current framework would mean requests that pass validation would then wait on completing the proxy request before proceeding to the next request.
  • implementing an additional worker pool to perform successful requests to the proxy asynchronously would introduce an additional layer of channel waits to the core engine, which may increase the burden on the go scheduler.

[Regala]

Hey @minight 😊

do you only want (non-wildcard) results to go through the proxy? or all requests

I would say non-wildcard. However, I could see -v debug (or alternative) forcing all requests to be proxied.

would you still use the feature if it caused scanning to run at 50% speed. Is the feature a higher priority than the speed of the scanning?

Definitely! I believe this is something people already assume is going to happen using some kind of middle man.

Thanks 💙

[joswr1ght]

Following up on @Regala's notes, I would also take advantage of a global --proxy option! Thanks!

[deruke]

I agree with the illustrious @joswr1ght , I often need to proxy to modify requests to contain authorization headers when I do content discovery.

[BreakfastSerial]

Proxying would be very welcome! For some pentest engagements, we need to keep a log of all requests and sometimes dynamically modify the requests via the proxy.