Open Mecanik opened 2 years ago
Hi @Mecanik!
It is possible now to set custom headers on the HTTP client level; you can do something like this:
<?php declare(strict_types=1);
use GuzzleHttp\Client;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\RequestOptions;
use Psr\Http\Message\RequestFactoryInterface;
use Psr\Http\Message\RequestInterface;
use Vantoozz\ProxyScraper\HttpClient\PsrHttpClient;
use function Vantoozz\ProxyScraper\proxyScraper;
require_once __DIR__ . '/vendor/autoload.php';
$httpClient = new PsrHttpClient(
new Client([
RequestOptions::CONNECT_TIMEOUT => 10,
RequestOptions::TIMEOUT => 20,
]),
new class implements RequestFactoryInterface {
public function createRequest(
string $method,
$uri
): RequestInterface {
return (new Request($method, $uri))
->withHeader('User-Agent', 'MyUserAgent')
->withHeader('Referer', 'https://example.com/');
}
}
);
$scraper = proxyScraper($httpClient);
This will add custom headers for all the requests
Awesome library!
Could you add support to add custom headers? It is quite a requirement to add custom user agent / referrer because many websites will block you from scraping too often.