nexus-stc / hyperboria

The Unlicense
105 stars 15 forks source link

Include the Internet Archive books on the bot pipeline #46

Open voidpunk opened 2 years ago

voidpunk commented 2 years ago

Some books present on the Internet Archive (IA) are not present on Libgen, I think it'd be nice to have them searchable through the bot as a fallback. Conceptually, the logic would be: query to bot -> if (book is on Libgen) {send it to user} else {query IA}. Then, if the book is present on IA, to retrieve it is just as easy as including in the codebase something like this: Archive.org-Downloader, which has a really fast and efficient downloader of borrow-only books from the IA. Thus, if the book is absent on Libgen and present on IA, the bot just download it from the IA and updates it on both Libgen and on Telegram for the querying user.

the-superpirate commented 2 years ago

@voidpunk

Sorry, the repo was paused for some time and I missed some letters.

The conceptual issue is that the bot has its own database. All queries goes to this database. It makes bot independent of external sources and more reliable.

It is required to import database of IA into Nexus database to make books available. But as far as I know, IA does not have publicly available dumps.