search

lapwinglabs / x-ray-phantom

phantom driver for x-ray.
112 stars 30 forks source link

Pagination with Login #2

Open JZL opened 9 years ago

JZL commented 9 years ago

Maybe this is specific to the site I am crawling but if a paginate a website after logging in, it loses the login (cookies/session I assume) on each successive new page

matthewmueller commented 9 years ago

Oh interesting, could you provide some psuedocode for context?

JZL commented 9 years ago 
xray('URL')
    .use(phantom())
//login(page # 1 )
    .wait("#barcodeOrUsername")
    .type('#barcodeOrUsername', 'USNER')
    .type("#password", "PASS")
    .click('input[type="submit"]')

//need to click a link, to get to pagination(page # 2)
    .wait(".menu-option-level-2")
    .click("a[href='/Mobile/MyAccount/ItemsOut']")
    .wait("table")

    .select("title")
    .paginate("#main > div > a:last-child[href]")
    .write(process.stdout)

The title is the right the first time, the logged clicked through page, but the next stdout is back to the page # 2

areida commented 9 years ago

I was having this exact same error today. For me the issue was that nightmare doesn't store cookies by default, you have to configure a path in the options.

.use(phantom({
    cookiesFile : '/path/to/cookies'
}))
JZL commented 9 years ago

I feel like that is probably where the error is but it doesn't work.

Thanks, though