A web scraper for scraping wuxiaworld. Written in python, using selenium and python cmd for an interactive shell experience with a command line utility to work with text along with a database to store information.
MIT License
2
stars
1
forks
source link
Page source being saved before the content loads #12
My guess is the code under the function that does 3 things (scrape, sleep, goto next page) is not doing things in proper order.
Maybe scrape -> then goto next page -> then sleep.
That way the content gets enough time to get loaded.
As you can see...
My guess is the code under the function that does 3 things (scrape, sleep, goto next page) is not doing things in proper order. Maybe scrape -> then goto next page -> then sleep. That way the content gets enough time to get loaded.