Closed GunnerGuyven closed 2 years ago
Thanks!
Auto-scroller, the premium account feature? I'd be careful using this tool with a paid account, as I mention in #40 if you hit the websites too much, they've started scrambling the contents per account.
But I was also scraping every interactive, so I'm not sure what their threshold is.
It was indeed the premium feature. I activated an account to do this testing because I was getting tired of waiting for the lockouts to lift.
The series of events were:
story.json
(chapters randomly missing in the middle, last_full_update
field present but unset)--force
to get the missing chapters. I found I couldn't scrape anything anymore.That last assumption was wrong, it was a consequence of going premium to do testing. Oh well.
Thanks for the warning. I'll make a throwaway to do scraping :)
These DOM changes were caused by the auto-scroller feature being enabled it should be disabled when scraping.
My apologies.