berstend / puppeteer-extra

💯 Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.48k stars 743 forks source link

using user-agent-override, navigator.platform gets unset on second navigation #175

Open cameronc56 opened 4 years ago

cameronc56 commented 4 years ago

Repro steps:

  1. Launch a puppeteer like so, you will need to replace the useragent, platform, and launch_options with something else. make sure to set the platform to something thats not your host:
const puppeteer = require("puppeteer-extra");                                                                           
// add stealth plugin and use defaults (all evasion techniques)                                                         
const stealth_plugin = require("puppeteer-extra-plugin-stealth");                                                       
const stealth = stealth_plugin();                                                                                       
// https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/evasions/user-agent-override/index.js
stealth.enabledEvasions.delete("user-agent-override");                                                                  
puppeteer.use(stealth); 
const UserAgentOverride = require("puppeteer-extra-plugin-stealth/evasions/user-agent-override");               
const ua = UserAgentOverride({userAgent: this.user_agent.toString(), locale: "en-US,en;q=0.9", platform: this.user_agent.data.platform});
puppeteer.use(ua);
this.browser = await puppeteer.launch(launch_options);                                                          
  1. open console. type window.navigator.platform and window.navigator.userAgent. Notice they are set to the values you passed into UserAgentOverride correctly.

  2. navigate to a page. repeat step 2. notice it is still set

  3. navigate to a different page. repeat step 2. notice window.navigator.platform gets reset to the system value. Further navigations do not set it back to the override value.

I have tried this with many different combinations of pages, and it always seems to get unset on the second navigation.

cameronc56 commented 4 years ago

It appears the user-agent-overrides get set "onPageCreated". Is there something we could change this to to make it cover both page creations and navigations?

cameronc56 commented 4 years ago
set_user_agent() {                                                                                                  
    this.page.on('framenavigated', () => {                                                                          
        this.page._client.send('Network.setUserAgentOverride', { userAgent: this.user_agent.toString(), 
acceptLanguage: 'en-US,en', platform: this.user_agent.data.platform });
    });                                                                                                             
};

this worked for me. it is not set upon launch (you just get a blank page anyways), but it sets it after any page navigations. it would be nice if the plugin could do this too

brunogaspar commented 4 years ago

Hey @cameronc56

Seems a nice fix, i would just send that as a pull request :), but unfortunately @berstend has been MIA (and i really hope he's okay) so your pull request would probably be a tad "stale" :(

I honestly wouldn't mind to maintain this library since i use it daily for various projects i have, but for now, it is what it is :)