To allow ProxyManager return null (or empty proxy) for not using a proxy for some specific requests

chhsiao90 commented 3 months ago

What kind of issue is this?

[ ] Question. This issue tracker is not the best place for questions. If you want to ask how to do something, or to understand why something isn't working the way you expect it to, use StackOverflow instead with the label 'stormcrawler': https://stackoverflow.com/questions/tagged/stormcrawler
[ ] Bug report. If you’ve found a bug, please include a test if you can, it makes it a lot easier to fix things. Use the label 'bug' on the issue.
[x] Feature request. Please use the label 'wish' on the issue.

In our use case, we have our own ProxyManager implementation, we'd like to have the flexibility to use proxies for some requests, but no proxy for other cases.

chhsiao90 commented 3 months ago

I'd like to create a PR for this.

jnioche commented 3 months ago

Yes please. ProxyManager could return an Optional depending on the Metadata. The code in the protocol implementations would need changing accordingly

apache / incubator-stormcrawler

To allow ProxyManager return null (or empty proxy) for not using a proxy for some specific requests #1246