Add new methods HttpLoader::useProxy() and
HttpLoader::useRotatingProxies([...]) to define proxies that the loader shall use. They can be used with a guzzle HTTP client instance (default) and when the loader uses the headless chrome browser. Using them when providing some other PSR-18 implementation will throw an exception.
(see https://github.com/crwlrsoft/crawler/issues/99)
Also, fix the HttpLoader::load() implementation won't throw any exception, because it shouldn't kill a crawler run. When you want any loading error to end the whole crawler execution
HttpLoader::loadOrFail() should be used. Also adapted the phpdoc in the LoaderInterface.
Add new methods
HttpLoader::useProxy()andHttpLoader::useRotatingProxies([...])to define proxies that the loader shall use. They can be used with a guzzle HTTP client instance (default) and when the loader uses the headless chrome browser. Using them when providing some other PSR-18 implementation will throw an exception. (see https://github.com/crwlrsoft/crawler/issues/99)Also, fix the
HttpLoader::load()implementation won't throw any exception, because it shouldn't kill a crawler run. When you want any loading error to end the whole crawler executionHttpLoader::loadOrFail()should be used. Also adapted the phpdoc in theLoaderInterface.