JimmXinu / FanFicFare

FanFicFare is a tool for making eBooks from stories on fanfiction and other web sites.
Other
754 stars 161 forks source link

Webnovel - Chapter contents are not being downloaded #825

Closed ghost closed 2 years ago

ghost commented 2 years ago

As it says in the title FFF stopped downloading contents of the chapters from webnovel. Everything else works perfectly, it gets synopsis/summary, tags, image, update date and so on and creates an epub but when I open the epub what I see is like below:

Usual info about the fic

Chapter 1 Chapter title

[No actual chapter text]

Chapter 2 Chapter title

[No actual chapter text]

Chapter 3 Chapter title

[No actual chapter text]

for however many chapters the fanfic has. Tried it with few fics randomly some I was already following some new, happened with all of them.

JimmXinu commented 2 years ago

The site has started putting the chapter text in javascript data objects instead of HTML.

It may or may not be feasible to parse it out in FFF. I'm looking into it.

JimmXinu commented 2 years ago

I've pushed a first cut fix (ff0a9a73354634343e2666bd504319e1c50fc8f7) and put test versions in the usual places.

I've only tested with a few chapters of a few stories. I won't be surprised if there are further issues.

CodeyAce95 commented 2 years ago

webnovel still not working

JimmXinu commented 2 years ago

It appears to be working for me in the test version. Did you follow the link and install the test version with:

pip install --extra-index-url https://testpypi.python.org/pypi --upgrade FanFicFare

Or from the mobileread forum if you use the plugin. You didn't mention which you use.

If you are using the test version, please provide a story URL that exhibits the problem--It's working with the ones I'm picking off the front page.

ghost commented 2 years ago

I tried the test plugin from mobileread when the first attempt at a fix was made, didn't work for me either. Example story URLs: https://www.webnovel.com/book/a-hero-arrives-to-orario-(percy-jackson-danmachi_22964156406510905 https://www.webnovel.com/book/a-bend-in-time-mina_22308746505503205

Example stories are short 2 and 5 chapters respectively but I tried few other stories from the below link as well with the same result. https://www.webnovel.com/stories/fanfic-book-literature?bookStatus=0&orderBy=5

JimmXinu commented 2 years ago

Looks like I uploaded an outdated PI version the previous time--sorry. I've updated and posted again to make sure it's up to date.

FYI, I tend to assume users opening issues here are CLI unless told otherwise.

ghost commented 2 years ago

Yep. It's working now. (v4.11.9) Thank you very much. Issue is solved so I'm closing it.

JimmXinu commented 2 years ago

Reason for reopening issue?

ghost commented 2 years ago

I've noticed if story/chapter contains the character "&" JSONDecodeError gets thrown. Tried it with both test plugin and CLI version. For example: fanficfare -b 178 -e 178 https://www.webnovel.com/book/frost's-greed-(marvel)_22100415205261005

I've tried looking for other characters that could similarly cause issues but couldn't find any other yet. Adding "&" to unescape regex in webnovel adaptor seems to fix it.

Sorry for re-opening the issue tbh I didn't know if I should re-open or create a new one but here we go.

JimmXinu commented 2 years ago

I thought there might be more escapes to be found.

Reopening is fine, I just thought you'd mis-clicked or something when there wasn't a new comment.

Test versions up.

CodeyAce95 commented 2 years ago

i am using the zip file from mobile read on the windows calibre app but maybe i got thr wrong file

CodeyAce95 commented 2 years ago

this is one thats not working https://www.webnovel.com/book/rebirth-of-the-mightiest-disciple-(reboot)_13520985605105905

CodeyAce95 commented 2 years ago

it works but i have to redownload the story

JimmXinu commented 2 years ago

What chapter does it fail at?

JimmXinu commented 2 years ago

I was able to download the story mentioned above without any apparent problems.

If you have specific problems you can point to, you can reopen this issue.