search

LucianoGanga / simple-headless-chrome

Simple abstraction to use Chrome as a Headless Browser with Node JS
MIT License
217 stars 50 forks source link

getCookies() #77

Open robkerry opened 7 years ago

robkerry commented 7 years ago

Does the getCookies() function show all Cookies loaded by Chrome? HTTP & JavaScript, main frame and iframes, 1st party and 3rd party?

I'm seeing far fewer Cookies than when using the actual Chrome browser.

If you visit any large e-commerce store or major website, it'll drop 100+ cookies, many from 3rd party domains for ad retargeting etc. Yet getCookies() returns no more than 30 cookies. I've even set wait() to 60 seconds and still don't get the full picture.

const HeadlessChrome = require('simple-headless-chrome')
const browser = new HeadlessChrome({
  headless: true
})
async function navigateWebsite () {
  try {
    await browser.init()
    const mainTab = await browser.newTab({ privateTab: false })
    await mainTab.goTo('http://www.marksandspencer.com/')
    await mainTab.wait(60000)
    await console.log( await mainTab.getCookies() )
    await browser.close()
  } catch (err) {
    console.log('ERROR!', err)
  }
}
navigateWebsite()

The code above gives me 24 cookies from .marksandspencer.com, .gwallet.com, .bluekai.com, .impdesk.com and .demdex.net.

But using regular Chrome gives me 127 cookies from .addthis.com, .adnxs.com, .adsrvr.org, .bidswitch.net, .bluekai.com, .cw.addthis.com, .demdex.net, .doubleclick.net, .dpm.demdex.net, .edrcdn.com, .edrcode.com, .facebook.com, .flashtalking.com, .google.co.uk, .google.com, .gwallet.com, .impdesk.com, .marksandspencer.com, .pixel.rubiconproject.com, .po.st, .rlcdn.com, .rubiconproject.com, .scorecardresearch.com, .univide.com, dpm.demdex.net, pixel.rubiconproject.com, www.google.co.uk and www.google.com.

Am I missing something?

StasToken commented 6 years ago

Hi, I assume that you are watching your browser, and when you visit through headless-chrome it creates a new profile, try to see the same site only through incognito mode? you should return the same number

I'm not saying anything - it's just my guess why it could be