dteviot / WebToEpub

A simple Chrome (and Firefox) Extension that converts Web Novels (and other web pages) into an EPUB.
Other
732 stars 139 forks source link

Please add site https://www.asianfanfics.com #584

Closed bichitoxxx closed 3 years ago

bichitoxxx commented 3 years ago

Please note, I'm basically the only developer working on WebToEpub, and I'm not paid for doing this. (WebToEpub is completely free, and generates no money.) By asking to add a site, you're asking me to give you some of my limited free time. So, I think it's not unreasonable for me to ask you to do as much as you can to help me.

Provide URL for web page that contains Table of Contents (list of chapters) of a typical story on the site

Did you try using the Default Parser for the site? If not, why not?

Instructions for using the default parser can be found at https://dteviot.github.io/Projects/webToEpub_DefaultParser.html

What settings did you use? What didn't work?

If the Default Parser did not work, if you have developer skills, did you try writing a new parser?

Instructions https://dteviot.github.io/Projects/webToEpub_FAQ.html#write-parser

If you don't have developer skills, can you ask a friend who does have them if they can do it for you?

If you tried writing a parser, and it doesn't work. Attach the parser here.

dteviot commented 3 years ago

@bichitoxxx Test versions for Firefox and Chrome have been uploaded to https://drive.google.com/drive/folders/1B_X2WcsaI_eg9yA-5bHJb8VeTZGKExl8?usp=sharing. Pick the one suitable for you, follow the "How to install from Source" instructions at https://github.com/dteviot/WebToEpub#how-to-install-from-source and let me know how it goes. Seems to work for

Note, you'll need to toggle the "Are you over 18" before WebToEpub can get "mature" pages. Otherwise you'll get errors like this

Error: Could not find content element for web page 'https://www.asianfanfics.com/story/view/990949/6/for-what-it-s-worth-part'. at chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/Parser.js:493:23 at async Promise.all (index 5) at async AsianfanficsParser.fetchWebPages (chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/Parser.js:462:17)

For my notes, took 40 minutes

bichitoxxx commented 3 years ago

Thank you very much it worked the first time, if there is any change during the tests I will notify you, for now I consider it valid and complete. the only fault that I see is that it does not take the name of the author Screenshot (2) Screenshot (3)

dteviot commented 3 years ago

@bichitoxxx Updated version (0.0.0.137) has been submitted to Firefox and Chrome stores. Firefox version is available now. Chrome might be available in 1 to 3 weeks.

bichitoxxx commented 3 years ago

Hello, I don't know if I should open another comment or it can be put here i'm having trouble downloading from asianfanfic, i get this every time i try a download Screenshot (2) https://www.asianfanfics.com/story/view/623145/matrimonio-por-contrato Screenshot (3)

dteviot commented 3 years ago

@bichitoxxx From the screen shots, it's failing to fetch chapter "Todo tiene una historia". Which I think is an "adult" chapter (based on the [M] the chapter name starts with.

You need to toggle the "Are you over 18" before WebToEpub can get "adult" pages. Otherwise you'll get errors like this

Error: Could not find content element for web page 'https://www.asianfanfics.com/story/view/990949/6/for-what-it-s-worth-part'. at chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/Parser.js:493:23 at async Promise.all (index 5) at async AsianfanficsParser.fetchWebPages (chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/Parser.js:462:17)

e.g. Try to browse to chapter 3, you should get a warning popup about adult content. Click the "I'm over 18" button to proceed. Then WebToEpub should work for those chapters.

bichitoxxx commented 3 years ago

Despite marking that I am over 18 years old, I still have problems with some chapters and the error Screenshot

After doing a second test by previously opening the chapters marked as M, continue like this. Screenshot (1)

https://www.asianfanfics.com/story/view/1455492/sleep-talk Here I make a third attempt with another story that is not rate M and the same problem, in fact there it does not download any, from the menu, only sometimes it downloads me without error if I do it by entering a chapter. something that did not happen to me before, I did not need to enter any chapter to download the story. in fact yesterday I downloaded without problems, it was today, it gave me a similar failure in Wattpad, as you will see, it also does not work on that platform.

asianfanfics

Screenshot (2)

Wattpad

Screenshot (3)

Screenshot (4)

As you will see, I also did the test in Sweek since in the previous two it no longer worked from the menu without entering the chapter, finding myself with the same problem. and since Sweek cannot be downloaded when entering the chapters, it has to be from the menu, it doesn't work.

Sweek

Screenshot (5)

dteviot commented 3 years ago

@bichitoxxx Provide URL of stories you're still getting the "Could not find content" errors for, and I'll look into it

"Unable to convert chapter" is a different problem. In short, web pages are HTML, an epub requires XHTML. WebToEpub will try to convert HTML to XHTML, but depending on how badly malformed the HTML is, it's not possible to automatically convert. So, you'd need to manually fix those pages. I recommend using Calibre.

dteviot commented 3 years ago

@bichitoxxx Something's strange. https://www.asianfanfics.com/story/view/1455492/sleep-talk is the ToC page of the story, it's not a chapter. (That's why WebToEpub can't find any chapter content.) It should not be in the list of chapters to get for the epub Have you checked "Add Page with Chapters to Chapters List" under the advanced options? If so, uncheck it.

bichitoxxx commented 3 years ago

https://www.asianfanfics.com/story/view/1455492/sleep-talk https://www.asianfanfics.com/story/view/990949/the-bunny-and-the-bear https://www.asianfanfics.com/story/view/584497/me-regresaste-a-la-vida https://www.wattpad.com/story/233115479-despu%C3%A9s-de-la-muerte-moonsun

I think I just discovered that the Wattpad error must have that option checked.

![Screenshot (1)](https://user-images.githubusercontent.com/29169926/132429097-be7b5337-d206-43ea-9ce0- Screenshot (2) f5c530588d29.png)

These are the stories that I have tried, but I think that the error may come from that option marked, because when unchecking it it has only failed me once in wattpad and none in Sweek, then I have been able to download without problems, but in asianfacfic it continues to fail with The rate M even if you mark over 18 years old, it does not download the story, it returns an error.

Screenshot (3)

dteviot commented 3 years ago

@bichitoxxx
The error message with "failed with network error Failed to fetch" means that when WebToEpub asked for the web page, there was no response at all. Usually means:

  1. A network problem
  2. Invalid URL (hostname in URL does not exist)
  3. Server is offline/faulted/no longer there

Note, in this case, looking at the URL and message, WebToEpub is trying to fetch an image that is hosted on a different site, (see "http://i.minus.com/" in the URL, which is different from asianfanfics). So, my guess is the site no longer exists. (You could try typing the URL into the browser directly and see what happens.)

However, as this is just fetching an image, it should not be a major problem. WebToEpub should just carry on, fetching the rest of the story. It's just that image will be missing.

And yes, turn off "Add Page with Chapters to Chapters List".

bichitoxxx commented 3 years ago

I have already tried it and it continues to give the same problem, the network is not or I know that it works without problems, what I have done is download from any chapter changing all the data by hand, Title, cover etc, why not download from the menu, does not download the correct data, and thus the complete book is downloaded without problem.

dteviot commented 3 years ago

@bichitoxxx There seems to be a translation error, I don't fully understand you.

  1. What's the URL of the story that always fails?
  2. What are you manually changing to make it work?
bichitoxxx commented 3 years ago

Hello, I have the problem in asianfanfic. The one involved with him unloading the lifros without problems, there are two.

https://www.asianfanfics.com/story/view/584497/me-regresaste-a-la-vida this book gave me error, 2 out of 5 download attempts. Screenshot

dteviot commented 3 years ago

@bichitoxxx OK, when I run that story I get errors similar to you. However, they all start with "Fetch of image". That means, the chapter had an image, but when WebToEpub tried to get the image to put on the page, it wasn't able to. This is not a critial error, WebToEpub will continue, it just won't have the image. Note, if you open the page itself, the normal web browser will also fail.

e.g. Looking at this error,

Error: Fetch of image 'http://images5.fanpop.com/image/photos/31500000/Y-Jessica-snsd-31548961-357-400.jpg' for page 'https://www.asianfanfics.com/story/view/584497/6/vivamos-juntas' failed with network error 404. at FetchImageErrorHandler.onResponseError (chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/HttpClient.js:32:25) at Function.checkResponseAndGetData (chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/HttpClient.js:166:45) at chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/HttpClient.js:160:31 at async Promise.all (index 0) at async AsianfanficsParser.fetchWebPages (chrome-extension://omliclippfibgegkdnanknpekbaahlhm/js/Parser.js:462:17)

The page with the probem is https://www.asianfanfics.com/story/view/584497/6/vivamos-juntas If I open than with a browser, you can scroll down, then see and image, followed by some text, then a missing image.

01-Browser screen capture

If I look at the HTML, I can see there instructions to show two images. The second image (which the browser isn't showing) is http://images5.fanpop.com/image/photos/31500000/Y-Jessica-snsd-31548961-357-400.jpg. This is the same image reported in the error.

02-HTML

So, WebToEpub is working as designed. It can't get the images and is telling you so.

bichitoxxx commented 3 years ago

Okay, it is a problem that sometimes occurs on its own and with the solutions that I tell you, it goes without problems. Thank you so much