JimmXinu / FanFicFare

FanFicFare is a tool for making eBooks from stories on fanfiction and other web sites.
Other
737 stars 156 forks source link

How to download Wattpad stories from Calibre #1072

Closed x-ekalfwons closed 1 month ago

x-ekalfwons commented 1 month ago

Hi!

I am trying to download Wattpad story in Calibre but it's showing the warning. I have attached the screenshot of it. I hope someone can help me.

LOG COPIED:

Status Title Author Comment URL Error Unknown Unknown Bad Story URL: (https://www.wattpad.com/story/276619166) for site: (www.wattpad.com) Example: (https://www.wattpad.com/story/9999999-story-title https://www.wattpad.com/story/9999999 https://www.wattpad.com/9999999-chapter-is-ok-too) https://www.wattpad.com/story/276619166

Link to Screenshot: https://drive.google.com/file/d/1xzHEQVnWHMKVJFls_C-dmpSOlwW1qhVM/view?usp=sharing

JimmXinu commented 1 month ago

I'm not seeing this happen with story URL https://www.wattpad.com/story/323944550

Do you have the latest FFF version installed?

x-ekalfwons commented 1 month ago

Yes. Please check the screeenshot: https://drive.google.com/file/d/1NVnQVuP0bgNSFqjaHtViIzfd6NFRIlSu/view?usp=sharing

x-ekalfwons commented 1 month ago

It's happening with every story that I am trynna download from Wattpad.

Also, can you please cut out the story's URL from your comment, I just picked a random book to test it out. I don't want to offend the Author of the book. I've edited my issue as well. Link for testing: https://www.wattpad.com/story/276619166

JimmXinu commented 1 month ago

Google first required I log in and now says access denied. You can attach the screen shot here.

x-ekalfwons commented 1 month ago

Screenshot (173) Screenshot (174)

JimmXinu commented 1 month ago

Run Calibre in debug mode (under Calibre's Preferences dropdown), try again, exit Calibre and post the debug log.

x-ekalfwons commented 1 month ago

It worked. Thank you so much for the help!

tuyetngan11 commented 1 month ago

Hello, I also have the same problem downloading stories from Wattpad as you. Can you show me a specific way to fix it? I tried running Caliber in debug mode but it still doesn't work, can you give me detailed instructions on how to fix it?

JimmXinu commented 1 month ago

Running in debug is not a fix--it's a debugging tool. Post the debug log so I have more details than "didn't work".

tuyetngan11 commented 1 month ago

LOG COPIED:

Status Title Author Comment URL Error Unknown Unknown Bad Story URL: (https://www.wattpad.com/story/364104597) for site: (www.wattpad.com) Example: (https://www.wattpad.com/story/9999999-story-title https://www.wattpad.com/story/9999999 https://www.wattpad.com/9999999-chapter-is-ok-too) https://www.wattpad.com/story/364104597

tuyetngan11 commented 1 month ago

Can you help me fix it? Thank you

JimmXinu commented 1 month ago

If you read the prior posts, you will see that x-ekalfwons posted exactly the same error message, which wasn't enough information then, and isn't now.

If you post a debug log of the failure, like I said before:

Run Calibre in debug mode (under Calibre's Preferences dropdown), try again, exit Calibre and post the debug log.

I will look at it. Otherwise, there's nothing I can do.

tuyetngan11 commented 1 month ago

sorry, but how can i show you the debug log? I'm not good at technology, sorry

tuyetngan11 commented 1 month ago

ApplicationPaletteChange event ignored Using calibre Qt style: True FFF: DEBUG: 2024-05-26 20:34:45,214: calibre_plugins.fanficfare_plugin.fff_plugin(215): Plugin FanFicFare macmenuhack file_path:C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt calibre Nhật kí gỡ lỗi calibre 7.11 embedded-python: True Windows-10-10.0.22621-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.22621') Python 3.11.5 Windows: ('10', '10.0.22621', 'SP0', 'Multiprocessor Free') Interface language: vi EXE path: C:\Program Files\Calibre2\calibre-debug.exe Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 11, 0) && Package KFX (from KFX Input) (2, 11, 0) && FanFicFare (4, 34, 1) && KFX metadata reader (from KFX Input) (2, 11, 0) && From KFX (2, 11, 0) && KFX Input (2, 11, 0) calibre 7.11 embedded-python: True Windows-10-10.0.22621-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.22621') Python 3.11.5 Windows: ('10', '10.0.22621', 'SP0', 'Multiprocessor Free') Interface language: vi EXE path: C:\Program Files\Calibre2\calibre-debug.exe Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 11, 0) && Package KFX (from KFX Input) (2, 11, 0) && FanFicFare (4, 34, 1) && KFX metadata reader (from KFX Input) (2, 11, 0) && From KFX (2, 11, 0) && KFX Input (2, 11, 0) QPA platform: windows devicePixelRatio: 1.5 logicalDpi: 96.0 x 96.0 physicalDpi: 105.21682847896439 x 105.10344827586206 [0.00] Starting up... [0.02] Showing splash screen... [0.16] splash screen shown [0.16] Initializing db... [0.53] db initialized [0.53] Constructing main UI... [2.34] GUI main window shown Key 'S' for shortcut Move to next match is already used by Lưu vào đĩa, ignoring Key 'G' for shortcut Lấy sách is already used by Thêm sách, ignoring Key 'S' for shortcut Move to next match is already used by Lưu vào đĩa, ignoring Key 'G' for shortcut Lấy sách is already used by Thêm sách, ignoring [2.41] main UI initialized... [2.41] Hiding splash screen FFF: DEBUG: 2024-05-26 20:36:56,269: configurable.py(1080): use_browser_cache: FFF: DEBUG: 2024-05-26 20:36:56,269: configurable.py(1100): use_basic_cache: FFF: DEBUG: 2024-05-26 20:36:56,269: fetcher_requests.py(114): ---------- REQ (GET) RequestsFetcher https://www.wattpad.com/apiv2/getcategories FFF: WARNING: 2024-05-26 20:39:04,713: adapter_wattpadcom.py(53): API_GETCATEGORIES failed. FFF: DEBUG: 2024-05-26 20:39:35,710: calibre_plugins.fanficfare_plugin.dialogs(493): self.extraoptions['anthology_url']:NOT FOUND FFF: DEBUG: 2024-05-26 20:39:35,711: calibre_plugins.fanficfare_plugin.fff_plugin(1150): FanFicFare v4.34.1 FFF: INFO: 2024-05-26 20:39:35,814: calibre_plugins.fanficfare_plugin.prefs(216): Using default settings FFF: DEBUG: 2024-05-26 20:39:35,829: configurable.py(1080): use_browser_cache: FFF: DEBUG: 2024-05-26 20:39:35,829: configurable.py(1100): use_basic_cache:true FFF: DEBUG: 2024-05-26 20:39:35,829: cache_basic.py(116): ========== MISS (GET) BasicCache https://www.wattpad.com/api/v3/stories/674163 FFF: DEBUG: 2024-05-26 20:39:35,829: fetcher_requests.py(114): ---------- REQ (GET) RequestsFetcher https://www.wattpad.com/api/v3/stories/674163 FFF: ERROR: 2024-05-26 20:41:41,927: calibre_plugins.fanficfare_plugin.dialogs(726): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Lỗi', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': 'Bad Story URL: (https://www.wattpad.com/story/674163) for site: (www.wattpad.com) Example: (https://www.wattpad.com/story/9999999-story-title https://www.wattpad.com/story/9999999 https://www.wattpad.com/9999999-chapter-is-ok-too)', 'url': 'https://www.wattpad.com/story/674163', 'site': 'www.wattpad.com', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://www.wattpad.com/story/674163[None-None]', 'listorder': 0, 'collision': 'Update EPUB if New Chapters'}:Bad Story URL: (https://www.wattpad.com/story/674163) for site: (www.wattpad.com) Example: (https://www.wattpad.com/story/9999999-story-title https://www.wattpad.com/story/9999999 https://www.wattpad.com/9999999-chapter-is-ok-too) Traceback (most recent call last): File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 699, in urlopen File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 382, in _make_request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 1010, in _validateconn File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connection.py", line 411, in connect File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\util\ssl.py", line 449, in ssl_wrapsocket File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\util\ssl.py", line 493, in _ssl_wrap_socket_impl File "ssl.py", line 517, in wrap_socket File "ssl.py", line 1108, in _create File "ssl.py", line 1379, in do_handshake ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\requests\adapters.py", line 439, in send File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 783, in urlopen File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 783, in urlopen File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 783, in urlopen [Previous line repeated 1 more time] File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\connectionpool.py", line 755, in urlopen File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\urllib3\util\retry.py", line 574, in increment urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.wattpad.com', port=443): Max retries exceeded with url: /api/v3/stories/674163 (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\adapter_wattpadcom.py", line 104, in doExtractChapterUrlsAndMetadata File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\requestable.py", line 119, in get_request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\requestable.py", line 111, in get_request_redirected File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\base_fetcher.py", line 133, in get_request_redirected File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\cache_basic.py", line 122, in fetcher_do_request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\decorators.py", line 102, in fetcher_do_request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\base_fetcher.py", line 106, in do_request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\fetchers\fetcher_requests.py", line 121, in request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\requests\sessions.py", line 542, in request File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\requests\sessions.py", line 655, in send File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\requests\adapters.py", line 516, in send requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.wattpad.com', port=443): Max retries exceeded with url: /api/v3/stories/674163 (Caused by ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "calibre_plugins.fanficfare_plugin.dialogs", line 713, in do_loop File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1356, in prep_download_loop File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1236, in get_story_metadata_only File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 325, in getStoryMetadataOnly File "C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\adapter_wattpadcom.py", line 108, in doExtractChapterUrlsAndMetadata fanficfare.exceptions.InvalidStoryURL: Bad Story URL: (https://www.wattpad.com/story/674163) for site: (www.wattpad.com) Example: (https://www.wattpad.com/story/9999999-story-title https://www.wattpad.com/story/9999999 https://www.wattpad.com/9999999-chapter-is-ok-too)

Is this correct?

tuyetngan11 commented 1 month ago

or this?

ApplicationPaletteChange event ignored Using calibre Qt style: True FFF: DEBUG: 2024-05-26 21:00:57,126: calibre_plugins.fanficfare_plugin.fff_plugin(215): Plugin FanFicFare macmenuhack file_path:C:\Users\TuyetNgan\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt calibre Nhật kí gỡ lỗi calibre 7.11 embedded-python: True Windows-10-10.0.22621-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.22621') Python 3.11.5 Windows: ('10', '10.0.22621', 'SP0', 'Multiprocessor Free') Interface language: vi EXE path: C:\Program Files\Calibre2\calibre-debug.exe Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 11, 0) && Package KFX (from KFX Input) (2, 11, 0) && FanFicFare (4, 34, 1) && KFX metadata reader (from KFX Input) (2, 11, 0) && From KFX (2, 11, 0) && KFX Input (2, 11, 0) calibre 7.11 embedded-python: True Windows-10-10.0.22621-SP0 Windows ('64bit', 'WindowsPE') ('Windows', '10', '10.0.22621') Python 3.11.5 Windows: ('10', '10.0.22621', 'SP0', 'Multiprocessor Free') Interface language: vi EXE path: C:\Program Files\Calibre2\calibre-debug.exe Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 11, 0) && Package KFX (from KFX Input) (2, 11, 0) && FanFicFare (4, 34, 1) && KFX metadata reader (from KFX Input) (2, 11, 0) && From KFX (2, 11, 0) && KFX Input (2, 11, 0) QPA platform: windows devicePixelRatio: 1.5 logicalDpi: 96.0 x 96.0 physicalDpi: 105.21682847896439 x 105.10344827586206 [0.00] Starting up... [0.02] Showing splash screen... [0.16] splash screen shown [0.16] Initializing db... [0.52] db initialized [0.52] Constructing main UI... [2.33] GUI main window shown Key 'S' for shortcut Move to next match is already used by Lưu vào đĩa, ignoring Key 'G' for shortcut Lấy sách is already used by Thêm sách, ignoring Key 'S' for shortcut Move to next match is already used by Lưu vào đĩa, ignoring Key 'G' for shortcut Lấy sách is already used by Thêm sách, ignoring [2.39] main UI initialized... [2.39] Hiding splash screen

JimmXinu commented 1 month ago

The developer who made the wattpad adapter used the wrong exception. Instead of InvalidStoryURL, it should report a connection error. Corrected in b01914c24ebbfbf11c33a54803bacd2fd3b8fbef

However, your fundamental problem is that the site is refusing your connection. I would speculate that you've done "too many" downloads in too short a time period. Many sites block people who make too many hits too fast, where both "too many" and "too fast" are unknown values.

Most times, you'll have to wait another unknown length of time for the block to clear.

When it does start working again, you can either download / update less (see FAQ), or try setting slow_down_sleep_time in personal.ini under [wattpad.com] to a value higher than 2.