Open johndoe-dev00 opened 3 years ago
This looks fine, but is there any reason to have this other than "it's a cool feature"? I'm a huge advocate of the "do one thing and do it well" philosophy, and I see this tool as a scraper of the blinkist material mainly, what's the advantage in lengthening this by visiting one extra page for each book just to get the Amazon asin? Even if put behind an optional flag, it add more logic and arguments to keep track of to the scrape_book_data
method and others.
Well, I guess my use case is a bit different, than the usual offline reading. I find the Blinkist Smartphone App pretty crappy in terms of deciding, which book to listen to next. So I want to listen to the books with the most amazon reviews. And with the Amazon IDs I gathered, I was able to scrape Amazon and generate a list of all blinkist books, ordered by number of amazon reviews:
Other people might have similar use cases, that involve a books Amazon ID, thats why I thought, I would create a pull requests. If you think this is out of scope of the intended use case, feel free to reject the pull request.
Use case Scrape Amazon product IDs from each book in order to later scrape the Amazon product pages for the review information.
Functionality I scraped the Amazon product IDs ("ASIN") from the "Buy" button on the https://www.blinkist.com/en/books/... pages. The ASIN is required to generate the Amazon product links
https://www.amazon.com/dp/<ASIN>
If available, the product IDs are stored in the *.json files. This feature needs to be enabled through the commandline switch--get-amazon-url
Also I added a "category_id" to the *.json files, that represents the index of the scraped category.