search

Corion / WWW-Mechanize-Chrome

automate the Chrome browser
https://metacpan.org/release/WWW-Mechanize-Chrome
Artistic License 2.0
30 stars 12 forks source link

Referer header not working #54

Open Rub3nCT opened 4 years ago

Rub3nCT commented 4 years ago

_addheader method is not working fine with Referer parameter, also tested with User-Agent and seems to be working fine.

$mech->add_header( Referer => $ref );

reload method is also crashing with ignoreCache option. Is working fine without ignoreCache option.

$mech->reload( ignoreCache => 1 );

Throws:

Uncaught exception from user code:
        Invalid parameters
        ignoreCache: boolean value expected
        -32602 at ... Chrome/DevToolsProtocol/Target.pm line 460

Test code attached comparing WWW::Mechanize and WWW::Mechanize::Chrome results. test.zip

Corion commented 4 years ago

Thank you very much for reporting the issues!

I think Chrome since v63+ does not send the Referer header anymore.

I'll look at the ignoreCache method and add a proper test for that.

Rub3nCT commented 4 years ago

Maybe this will help with Referer header:

Starting from Chrome 79, request header modifications affect Cross-Origin Resource Sharing (CORS) checks. If modified headers for cross-origin requests do not meet the criteria, it will result in sending a CORS preflight to ask the server if such headers can be accepted. If you really need to modify headers in a way to violate the CORS protocol, you need to specify 'extraHeaders' in opt_extraInfoSpec. On the other hand, response header modifications do not work to deceive CORS checks. If you need to deceive the CORS protocol, you also need to specify 'extraHeaders' for the response modifications.

Starting from Chrome 79, the webRequest API does not intercept CORS preflight requests and responses by default. A CORS preflight for a request URL is visible to an extension if there is a listener with 'extraHeaders' specified in opt_extraInfoSpec for the request URL. onBeforeRequest can also take 'extraHeaders' from Chrome 79.

Starting from Chrome 79, the following request header is not provided and cannot be modified or removed without specifying 'extraHeaders' in opt_extraInfoSpec:

Starting from Chrome 72, if you need to modify responses before Cross Origin Read Blocking (CORB) can block the response, you need to specify 'extraHeaders' in opt_extraInfpSpec.

Starting from Chrome 72, the following request headers are not provided and cannot be modified or removed without specifying 'extraHeaders' in opt_extraInfoSpec:

Starting from Chrome 72, the Set-Cookie response header is not provided and cannot be modified or removed without specifying 'extraHeaders' in opt_extraInfoSpec.

Source: https://developer.chrome.com/extensions/webRequest

Corion commented 4 years ago

Thank you for the link, but I don't see how it applies to the situation. The Referer header is not settable/addable since at least Chrome version 63, so the linked documentation does not seem to apply to the situation.

Corion commented 4 years ago

See https://bugs.chromium.org/p/chromium/issues/detail?id=849972 for the corresponding Chromium bug