Stuck at the same page

[rivernews]

Supervisor job params:

{
  "scraperJobRequestData": {
    "pubsubChannelName": "scraperJobChannel:https://www.glassdoor.com/Overview/Working-at-Mozenda-EI_IE674543.11,18.htm:0:startAtPage0",
    "orgInfo": "https://www.glassdoor.com/Overview/Working-at-Mozenda-EI_IE674543.11,18.htm"
  }
}

Feels like it fail to proceed to next page, so just keep scraping the same page for 4 times, so reach 40, but only processed 10 actually.

Some ideas to tackle this forever-stuck issue:

• [ ] Make sure for each round of while loop, the page number in the webpage increments.
• [ ] Examine the element-based termination condition
  • find link and click failed, what should we do
  • locate review panel failed, what should we do

Might be of same root problem as #81 .

[rivernews]

Looks like similar root cause as #81 . Will honor #81 and work there instead and close this ticket.