dipu-bd / lightnovel-crawler

Generate and download e-books from online sources.
https://pypi.org/project/lightnovel-crawler/
GNU General Public License v3.0
1.42k stars 279 forks source link

[madara template] Detect volumes and premium-block #1642

Closed idMysteries closed 2 weeks ago

idMysteries commented 1 year ago

some sites on this template have their own volumes.

https://mangamammy.ru/manga/falshivaya-krasota/ div.listing-chapters_wrap ul.volumns > li -> 2 volumes

https://mangarockteam.com/manga/heroes-demons-villains/ div.listing-chapters_wrap ul.no-volumn > li -> 19 chaps

With volumes: "div.listing-chapters_wrap ul.volumns > li" -> list of "li" tag is volumes Without volumes: "div.listing-chapters_wrap ul.no-volumn > li" -> list of "li" tag is chaps


And as you can see, there may not be sorted volumes here. Therefore, we need to sort the volumes if ul.volumns.

https://github.com/dipu-bd/lightnovel-crawler/pull/1568/commits/2ee60fd7ea156adf7a3d3bebee61cd5e07e5e1cd


https://github.com/VendingMp/Madara-Child-ZAL/blob/main/madara-child-zal/madara-core/single/info-chapters.php Here is madara template for chap As you can see in php ul.no-volumn and ul.volumns IS UNIVERSAL RULE

$single = isset( $manga['0']['chapters'] ) ? $manga['0']['chapters'] : null;
<ul class="main version-chap <?php echo ($single ? 'no-volumn':'volumns');?>">

Originally posted by @idMysteries in https://github.com/dipu-bd/lightnovel-crawler/issues/1568#issuecomment-1258189221

idMysteries commented 1 year ago

Each volume has a back-sorted list of chapters. But the volumes order themselves may not be sorted.

4 3 1 2 volumes

idMysteries commented 1 year ago

And premium block .wp-manga-chapter:not(.premium-block) > a

From madara php sources: if not bought -> class premium-block

image

for a in reversed(soup.select("ul.no-volumn .wp-manga-chapter:not(.premium-block) > a")):
dipu-bd commented 1 year ago

Each volume has a back-sorted list of chapters. But the volumes order themselves may not be sorted.

That's how the world is. Nothing is easy.

dipu-bd commented 1 year ago

We can make a complain to the site to fix their order of volumes if it works

idMysteries commented 1 year ago

Dear site developers. We have developed a parser for your site so that you do not receive money from advertising, but we noticed that the order of volumes is not correct. Please fix it.