search

nesaku / BiblioReads

An Alternative Private Goodreads Front-End.
https://biblioreads.eu.org
GNU Affero General Public License v3.0
183 stars 9 forks source link

[Feature] Scrape All Books By Author #11

Closed brucekinsey closed 1 year ago

brucekinsey commented 1 year ago

Is your feature request related to a problem/an inconvenience? Please describe. Whenever you scrape an author, the user only gets 10 results. I'd like to be able to scrape more (preferably all) books for the author, but only upon request.

Describe the solution/feature you'd like Currently you can POST to /api/author-scraper with "body": "{\"queryURL\":\"https://www.goodreads.com/author/show/3389.Stephen_King\"}". I propose either:

a) list-scraper.js could accept a parameter, like authorList = true, where this then modifies the find in line 27 to "td > a > img.bookCover" b) POST to something like /api/author-list-scraper with "body": "{\"queryURL\":\"https://www.goodreads.com/author/list/3389.Stephen_King?page=1&per_page=100\"}". I believe this could be done with a small change to /api/list-scraper.js, since all other page elements match: line 27: .find("td > div.js-tooltipTrigger.tooltipTrigger > a > img") to .find("td > a > img.bookCover")

Describe alternatives you've considered I tried posting the queryURL to author-scraper, book-scraper, and list-scraper. All of them fail, due to the fact they aren't setup for this.

nesaku commented 1 year ago

Hello brucekinsey,

Thank you for creating this feature request. This has now been added in https://github.com/nesaku/BiblioReads/commit/b38728985d21838821940e9ad002b7bc293486fe (v2.15.0), all books for the author can now be shown.