JimmXinu / FanFicFare

FanFicFare is a tool for making eBooks from stories on fanfiction and other web sites.
Other
774 stars 165 forks source link

wuxiaworld.com adapter not working after website redesign #796

Closed mtsitzer closed 2 years ago

mtsitzer commented 2 years ago

The adapter for wuxiaworld.com is not working after the recent website redesign

JimmXinu commented 2 years ago

This has also been reported on MobileRead.

This is want I said about it there:

I'm not sure that is fixable. The site is now requiring your browser to download and run a sizeable obfuscated javascript file. And another 100+ other JS files??

FFF doesn't have a javascript engine, and just using FlareSolverr proxy doesn't work either.

mtsitzer commented 2 years ago

Roger that. That does indeed sound like a dead end

vuonojenmustaturska commented 2 years ago

Wuxiaworld seems to be doing the heavy lifting using some sort of a bespoke binary grpc protocol, for example chapters are fetched by posting a very small struct to api.wuxiaworld.com/wuxiaworld.api.v2.Chapters/GetChapter (content type: application/grpc-web+proto) which returns a struct that contains the chapter title and content html along with other stuff (cover urls etc). At a glance it seems fairly barebones, the problem where I stopped was that browser dev tools (and har files they produce) do not handle binary requests gracefully and doing the capture outside a browser is fairly annoying for https

budikesuma commented 2 years ago

@JimmXinu

Chapter Content fetch:

<div class="chapter-content ww
<div class="fr-view

Chapter Title fetch:

<h4 class="MuiTypography-root MuiTypography-h4 my-[20px] flex items-center

Uploaded file:

Wuxiaworld.com Chapters.zip

Good luck devs...

JimmXinu commented 2 years ago

FYI, unless someone steps forward with a PR, this site will be removed in the next version, sometime Mar 2022.

Chibba-dev commented 2 years ago

this https://github.com/dipu-bd/lightnovel-crawler works on ww.com again