FriendsOfPHP / Goutte

Goutte, a simple PHP Web Scraper
MIT License
9.26k stars 1.01k forks source link

Use User-Agent header from Guzzle client #381

Open 4n70w4 opened 5 years ago

4n70w4 commented 5 years ago

Guzzle code:

$base_uri = 'https://google.com/';

$headers = [
        'User-Agent' => $useragent,
];

$transport= new Client(['base_uri' => $base_uri, RequestOptions::HEADERS => $headers]);

$transport->request('GET', 'http://webhook.site/afb7fc90-9666-4e39-947c-35fb11646755');

Goutte code:

$client = new \Goutte\Client();
$client->setClient($transport);

$client->request('GET', 'http://webhook.site/afb7fc90-9666-4e39-947c-35fb11646755');

Guzzle result:

host: webhook.site
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36

Goutte result:

host: google.com
user-agent: Symfony BrowserKit
bilalghouri commented 4 years ago

same problem, goutte is ignoring the guzzle client completely (ignores the cookie jar as well)

sunnysideup commented 4 years ago

try:

new \GuzzleHttp\Client(['headers' => ['User-Agent' => null]]);
sunnysideup commented 4 years ago

see: https://github.com/guzzle/guzzle/issues/894

4n70w4 commented 4 years ago

@sunnysideup what for?

sunnysideup commented 4 years ago

Ignore my comments, I thought it was helpful, but perhaps it is not.

gadnis commented 3 years ago

@4n70w4 @bilalghouri Hey! Did you find a solution?

It's 2020, but I have same problem with "guzzlehttp/guzzle": "^6.3", and "fabpot/goutte": "^3.2",. I know that there is goutte v.4 available now, but it uses symfony httpClient, but it's new and there's no time for refactoring my code unfortunately..

bavial commented 3 years ago

try: $transport= new Client(['HTTP_USER_AGENT' => 'Mozilla/5.0]);