segment-boneyard / nightmare

A high-level browser automation library.
https://open.segment.com
19.54k stars 1.08k forks source link

how to do save multiple page data in once?Please #1619

Open nijun008 opened 3 years ago

nijun008 commented 3 years ago

I want to... ,open a page, get the page data, then click next page, continue to get page data, all the way to the target page, my code for now

nightmare
  .goto(`https://www.appinn.com/`)
  .wait('.latestPost')
  .wait(() => {
    window.htmlArr = window.htmlArr || []

    if (!document.querySelector('.latestPost')) {
      return false
    }

    window.htmlArr.push(document.querySelector('#latest-posts').innerHTML)

    let currentPage = document.querySelector('.page-numbers.current')

    if (currentPage && currentPage.innerHTML == 5) {
      return true
    }

    let nextBtn = document.querySelector(".nav-links .next")
    if (nextBtn) {
      nextBtn.click()
      return false
    }
    return true
  })
  .evaluate(() => {
    return window.htmlArr
  })
  .end()
  .then(htmlList => {
    conosle.log(htmlList )
  })

htmlList only last page data, none others page data

What should I do? thank you.