benediktwerner / NovelDownloader

0 stars 1 forks source link

Not Working with Qidan or WuxiaWorld #1

Closed RuthlessRuler closed 5 years ago

RuthlessRuler commented 5 years ago

In qidan.py the url is wrong. (webnovels instead of webnovel)

` C:\Users\Users\Downloads\NovelDownloader-master>python main.py Welcome to BND (Bene's Novel Downloader) Enter a book or '?' to list all current books:

? Traceback (most recent call last): File "main.py", line 124, in main() File "main.py", line 85, in main book = choose_book() File "main.py", line 20, in choose_book utils.list_books() File "C:\Users\Users\Downloads\NovelDownloader-master\utils.py", line 130, in list_books for book in sorted(os.listdir(BASE_DIR)): FileNotFoundError: [WinError 3] The system cannot find the path specified: 'books'

C:\Users\Users\Downloads\NovelDownloader-master>python main.py Welcome to BND (Bene's Novel Downloader) Enter a book or '?' to list all current books:

?

No book entered. Bye.

C:\Users\Users\Downloads\NovelDownloader-master>python main.py Welcome to BND (Bene's Novel Downloader) Enter a book or '?' to list all current books:

Battle Through The HEavens

The book 'Battle Through The HEavens' does not exist. Do you want to create it? (Y/n) Y

New book. Creating directory books\battle through the heavens Creating new config for Battle Through The HEavens: [0] Custom [1] qidian [2] wuxiaworld Website: 2 Traceback (most recent call last): File "main.py", line 124, in main() File "main.py", line 89, in main utils.ensure_config(book) File "C:\Users\Users\Downloads\NovelDownloader-master\utils.py", line 52, in ensure_config config.create_config(book) File "C:\Users\Users\Downloads\NovelDownloader-master\config.py", line 31, in create_config config["website"] = _get_website() File "C:\Users\Users\Downloads\NovelDownloader-master\config.py", line 14, in _get_website return websites.WEBSITES[website_index] IndexError: list index out of range

C:\Users\Users\Downloads\NovelDownloader-master>python downloader.py

C:\Users\Users\Downloads\NovelDownloader-master>python utils.py

C:\Users\Users\Downloads\NovelDownloader-master>python main.py Welcome to BND (Bene's Novel Downloader) Enter a book or '?' to list all current books:

? battle through the heavens

1

The book '1' does not exist. Do you want to create it? (Y/n) n Ok. Bye.

C:\Users\Users\Downloads\NovelDownloader-master>python main.py Welcome to BND (Bene's Novel Downloader) Enter a book or '?' to list all current books:

BTTH

The book 'BTTH' does not exist. Do you want to create it? (Y/n) Y

New book. Creating directory books\btth Creating new config for BTTH: [0] Custom [1] qidian [2] wuxiaworld Website: 1 Book id? 8093990805004205 Name? (optional) Add chapter titles? (y/N) Y Config created at: books\btth\config.yml

Traceback (most recent call last): File "main.py", line 124, in main() File "main.py", line 90, in main conf = config.load_config(book) File "C:\Users\Users\Downloads\NovelDownloader-master\config.py", line 60, in load_config config["website"] = websites.from_config(config["website"]) File "C:\Users\Users\Downloads\NovelDownloader-master\websites__init__.py", line 83, in from_config "toc_url": config["toc_url"], TypeError: 'type' object is not subscriptable

C:\Users\Users\Downloads\NovelDownloader-master>`

benediktwerner commented 5 years ago

Yeah, the last changes introduced some major bugs I didn't notice. Should be fixed now.

Delete the books directory, get the new version and try again.

I also added some more instructions and help.

RuthlessRuler commented 5 years ago

Wuxiaworld is throwing an error:

`Welcome to BND (Bene's Novel Downloader) Enter a book or '?' for help:

BTTH

The book 'BTTH' does not exist. Do you want to create it? (Y/n) Y

New book. Creating directory books\btth Creating new config for BTTH: [0] Custom [1] qidian [2] wuxiaworld Website: 2 Book id? battle-through-the-heavens Name? (optional) Add additional chapter titles? (y/N) Y Config created at: books\btth\config.yml

Selected BTTH No chapters on disk

Do you want to download chapters? (Y/n) Y Which chapters do you want to download? First chapter? 1 Last chapter? 1648

Downloading 0 of 1648 N/A% [ ] ETA: --:--:-- Traceback (most recent call last): File "main.py", line 156, in main() File "main.py", line 138, in main downloaded_chapters = download_chapters(conf) File "main.py", line 84, in download_chapters downloader.download_chapters(chapter_start, chapter_end, conf) File "C:\Users\USer\Downloads\NovelDownloader-master\downloader.py", line 56, in download_chapters download_chapters(chapter_start, chapter_end, config) File "C:\Program Files\Python37\lib\asyncio\base_events.py", line 568, in run_until_complete return future.result() File "C:\Users\USer\Downloads\NovelDownloader-master\downloader.py", line 49, in download_chapters await asyncio.gather(*tasks) File "C:\Users\USer\Downloads\NovelDownloader-master\downloader.py", line 14, in download_chapter content = await config["website"].download_chapter(chapter, config, session) File "C:\Users\USer\Downloads\NovelDownloader-master\websites__init__.py", line 22, in download_chapter url = cls.get_chapter_url(chapter, config) File "C:\Users\USer\Downloads\NovelDownloader-master\websites\wuxiaworld.py", line 39, in get_chapter_url return Wuxiaworld.url + cls.tocs[config["book_id"]][chapter] AttributeError: type object 'Wuxiaworld' has no attribute 'url' Downloading 1648 of 1648 100% [#######################################################################] Time: 0:00:00`

While Webnovel download perfectly but there is problem in the parsing of the HTML Webpage. I.E: The Page has no paragraphs formatting & "" is replaced by “ & ” Respectively in HTML Convert and the downloaded HTML. In Case of converting to TXT Files, the "" are there as it is and formatting is there to some extent.(Still Breaks after each para is missing and ETC.

Even though Chapter title Option is marked as Yes, only Chapter 1 and Chapter 2 are displayed instead of the Chapter Title.(In txt file conversion)

And Last but not least add the ability to convert the downloaded chapters in to EPUB too.(Like this script does: https://github.com/SGOrava/novelDownloader).

benediktwerner commented 5 years ago

Ok, the wuxiaworld thing should also be fixed now, that was really stupid, I really should write some tests 😅

But Webnovel is working without problems for me. I guess that might be an encoding issue, so I forced the encoding for all downloads to utf-8 now, maybe that helps.

The add chapter title is supposed to work like that. It only adds generic numbered titles and I use it in case a novel doesn't have titles or they aren't downloaded (like on Webnovel). Maybe I will add titles to webnovel novels later.

As for epubs, I might do that at some point, but I don't think it's really worth the effort as you can just read txt-files on Kindles and it works fine. You don't get any fancy formatting, but that's not really neccessary 99% of the time.