Webnovel error download history

[tom221-e]

Descargar 1 libro(s) de «fanfiction» FFF: INFO: 2022-04-22 22:06:27,661: calibre_plugins.fanficfare_plugin.jobs(49): CPUs:3 FFF: INFO: 2022-04-22 22:06:27,662: calibre_plugins.fanficfare_plugin.jobs(52): FanFicFare v4.11.12 FFF: INFO: 2022-04-22 22:06:27,662: calibre_plugins.fanficfare_plugin.jobs(68): Lanzar proceso en segundo plano para www.webnovel.com: https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305 FFF: INFO: 2022-04-22 22:06:30,534: calibre_plugins.fanficfare_plugin.jobs(130):

================================================================================ site:(www.webnovel.com) FFF: INFO: 2022-04-22 22:06:28,965: calibre_plugins.fanficfare_plugin.jobs(211):

-------------------------------------------------------------------------------- https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305 FFF: DEBUG: 2022-04-22 22:06:28,982: configurable.py(1035): use_browser_cache: FFF: DEBUG: 2022-04-22 22:06:28,983: configurable.py(1049): use_basic_cache:true FFF: DEBUG: 2022-04-22 22:06:28,986: adapter_webnovelcom.py(85): https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305 FFF: DEBUG: 2022-04-22 22:06:28,988: fetcher.py(234): ========== MISS (GET) BasicCache https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system%21_15693570606776305 FFF: DEBUG: 2022-04-22 22:06:28,988: fetcher.py(469): ---------- REQ (GET) RequestsFetcher https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system%21_15693570606776305 FFF: DEBUG: 2022-04-22 22:06:30,030: fetcher.py(482): response code:200 FFF: DEBUG: 2022-04-22 22:06:30,031: requestable.py(50): Encoding:utf8 FFF: INFO: 2022-04-22 22:06:30,364: calibre_plugins.fanficfare_plugin.jobs(439): Exception: {'title': 'Reincarnated as a dragon egg in DxD with a Fate System!', 'author_sort': 'GreatSage_Master1', 'author': ['GreatSage_Master1'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': 1395, 'begin': None, 'end': None, 'comment': "'NoneType' object has no attribute 'get_text'", 'url': 'https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305', 'site': 'www.webnovel.com', 'series': '', 'added': False, 'pubdate': None, 'publisher': 'www.webnovel.com', 'listorder': 0, 'collision': 'Actualizar EPUB si hay nuevos capítulos', 'uniqueurl': 'https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305[None-None]', 'is_adult': False, 'username': 'NoneGiven', 'password': '', 'icon': 'dialog_error.png', 'epub_for_update': 'C:\Users\TomasE\AppData\Local\Temp\calibre_8ijklkcj\fanficfare_twgdf4dz\old-1395-tfl5r82t.epub', 'calibre_columns': {}, 'outfile': 'C:\Users\TomasE\AppData\Local\Temp\calibre_8ijklkcj\fanficfare_twgdf4dz\bgmeta-s4c6gm2p.epub'}:'NoneType' object has no attribute 'get_text' Traceback (most recent call last): File "calibre_plugins.fanficfare_plugin.jobs", line 260, in do_download_for_worker File "C:\Users\TomasE\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 308, in getStoryMetadataOnly File "C:\Users\TomasE\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\adapter_webnovelcom.py", line 109, in doExtractChapterUrlsAndMetadata File "C:\Users\TomasE\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\htmlcleanup.py", line 71, in stripHTML AttributeError: 'NoneType' object has no attribute 'get_text'

FFF: INFO: 2022-04-22 22:06:30,535: calibre_plugins.fanficfare_plugin.jobs(142): Resultado de la descarga: Error https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305 'NoneType' object has no attribute 'get_text'

FFF: INFO: 2022-04-22 22:06:30,535: calibre_plugins.fanficfare_plugin.jobs(166): Error https://www.webnovel.com/book/reincarnated-as-a-dragon-egg-in-dxd-with-a-fate-system!_15693570606776305

[ghost]

FFF can't find some metadata due to site changes. I tried changing adapter_webnovelcom.py lines between 103 to 131 from this:

        title = bookdetails.find('h2')
        # done as a loop incase there isn't one, or more than one.
        for tag in title.find_all('small'):
            tag.extract()
        self.story.setMetadata('title', stripHTML(title))

        detail_txt = stripHTML(bookdetails.find('p', {'class': re.compile('detail')}))
        if "Completed" in detail_txt:
            self.story.setMetadata('status', 'Completed')
        else:
            self.story.setMetadata('status', 'In-Progress')

        meta_tag = bookdetails.find('address').p
        meta_txt = stripHTML(meta_tag)

        def parse_meta(mt,label,setmd):
            if label in mt:
                data = mt.split(label,1)[1].split('Translator:', 1)[0].split('Editor:', 1)[0].strip()
                if data:
                    # print("setting %s to %s"%(setmd, data))
                    self.story.setMetadata(setmd, data)

        parse_meta(meta_txt,'Author:','author')
        self.story.setMetadata('authorId', self.story.getMetadata('author'))
        # There is no authorUrl for this site, so I'm setting it to the story url
        # otherwise it defaults to the file location
        self.story.setMetadata('authorUrl', url)
        parse_meta(meta_txt,'Translator:','translator')
        parse_meta(meta_txt,'Editor:','editor')

to this:

        title = bookdetails.find('h1')
        # done as a loop incase there isn't one, or more than one.
        for tag in title.find_all('small'):
            tag.extract()
        self.story.setMetadata('title', stripHTML(title))
        # I couldn't find any tag containing info about story status
        # so I set it to blank
        self.story.setMetadata('status', '')

        # Author details. Name, id, url...
        autdet = bookdetails.find('a', {'class': re.compile('c_primary')})

        self.story.setMetadata('author',stripHTML(autdet))

        self.story.setMetadata('authorId',re.search(r"/([0-9]+)", autdet.get("href"))[1])

        self.story.setMetadata('authorUrl', "https://www.webnovel.com" + autdet.get("href"))

It works but I'm unsure about somethings such as, did site use to have (perhaps still has) translator and editor fields apart from author, if I'm not mistaken original code suggest that was (maybe still is) the case. I couldn't find any tag that contains info about stories completion status. I'm uncertain about site tags, they do get added under the summary in epub but tags field in calibre is empty. So I don't know if the tags are found successfully and added under the summary or summary already contained "Site Tag:" portion. Anyway, I'm not confident I understand enough about FFF to make a PR but hope this helps, little as it may be.

[CodeyAce95]

does anyone know how to fix it on windows version of calibre

[JimmXinu]

There definitely are stories that have translator and editor fields and author without a link (examples 1 2). Second is also an example of a Completed story for status.

@CarelessLazy, thanks for your help; I've taken part of it for the fix.

[JimmXinu]

Test versions posted in the usual places.

[gille90]

got same error at first, got a new one since I updated to new test version. So i'll post it here

'_sre.SRE_Match' object has no attribute 'getitem' https://www.webnovel.com/book/i-write-what-i-want._20920678205043705

[JimmXinu]

@gille90 - I'm not seeing that problem. Can you run FFF in debug and provide the more detailed log?

[gille90]

calibre Foutopsporingslog calibre 3.48 [64bit] embedded-python: True is64bit: True Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE') ('Windows', '7', '6.1.7601') Python 2.7.15+ Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free') Interface language: nl Successfully initialized third party plugins: FanFicFare (4, 11, 13) calibre 3.48 [64bit] embedded-python: True is64bit: True Windows-7-6.1.7601-SP1 Windows ('64bit', 'WindowsPE') ('Windows', '7', '6.1.7601') Python 2.7.15+ Windows: ('7', '6.1.7601', 'SP1', u'Multiprocessor Free') Interface language: nl Successfully initialized third party plugins: FanFicFare (4, 11, 13) Turning on automatic hidpi scaling devicePixelRatio: 1.0 logicalDpi: 96.0 x 96.0 physicalDpi: 72.035450517 x 72.0 Using calibre Qt style: True [0.00] Starting up... [0.00] Showing splash screen... [0.05] splash screen shown [0.05] Initializing db... [0.11] db initialized [0.11] Constructing main UI... DEPRECATION: The OpenSSL being used by this python install (OpenSSL 1.0.2q 20 Nov 2018) does not meet the minimum supported version (>= OpenSSL 1.1.1) in order to support TLS 1.3 required by Cloudflare, You may encounter an unexpected Captcha or cloudflare 1020 blocks. Loaded cached store plugin for: Kobo at version: 8 Loaded cached store plugin for: Project Gutenberg at version: 8 Loaded cached store plugin for: ManyBooks at version: 2 Loaded cached store plugin for: XinXii at version: 2 FFF: DEBUG: 2022-04-26 22:52:41,039: calibre_plugins.fanficfare_plugin.fff_plugin(218): Plugin FanFicFare macmenuhack file_path:C:\Users\francy\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt [2.12] main UI initialized... [2.12] Hiding splash screen [12.23] splash screen hidden [12.23] Started up in 12.23 seconds with 982 books FFF: DEBUG: 2022-04-26 22:53:18,006: calibre_plugins.fanficfare_plugin.fff_plugin(1129): FanFicFare v4.11.13 FFF: DEBUG: 2022-04-26 22:53:18,154: configurable.py(1035): use_browser_cache: FFF: DEBUG: 2022-04-26 22:53:18,155: configurable.py(1049): use_basic_cache:true FFF: DEBUG: 2022-04-26 22:53:18,161: adapter_webnovelcom.py(85): https://www.webnovel.com/book/i-write-what-i-want._20920678205043705 FFF: DEBUG: 2022-04-26 22:53:18,161: fetcher.py(234): ========== MISS (GET) BasicCache https://www.webnovel.com/book/i-write-what-i-want._20920678205043705 FFF: DEBUG: 2022-04-26 22:53:18,161: fetcher.py(469): ---------- REQ (GET) RequestsFetcher https://www.webnovel.com/book/i-write-what-i-want._20920678205043705 FFF: DEBUG: 2022-04-26 22:53:18,726: fetcher.py(482): response code:200 FFF: DEBUG: 2022-04-26 22:53:18,727: requestable.py(50): Encoding:utf8 FFF: ERROR: 2022-04-26 22:53:19,059: calibre_plugins.fanficfare_plugin.dialogs(714): Exception: {u'comments': u'', u'end': None, u'author': [u'Ryan_Colman'], u'status': u'Fout', u'showerror': True, u'series': u'', u'comment': u"'_sre.SRE_Match' object has no attribute 'getitem'", u'title': u'I write what I want.', u'begin': None, u'listorder': 0, u'pubdate': None, u'added': False, u'calibre_id': 1054, u'good': False, u'author_sort': u'Ryan_Colman', u'site': 'www.webnovel.com', u'collision': u'EPUB bijwerken bij nieuwe hoofdstukken', u'publisher': u'www.webnovel.com', u'url': u'https://www.webnovel.com/book/i-write-what-i-want._20920678205043705'}:'_sre.SRE_Match' object has no attribute 'getitem' Traceback (most recent call last): File "calibre_plugins.fanficfare_plugin.dialogs", line 701, in do_loop File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1343, in prep_download_loop File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1215, in get_story_metadata_only File "C:\Users\francy\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\base_adapter.py", line 308, in getStoryMetadataOnly File "C:\Users\francy\AppData\Roaming\calibre\plugins\FanFicFare.zip\fanficfare\adapters\adapter_webnovelcom.py", line 129, in doExtractChapterUrlsAndMetadata TypeError: '_sre.SRE_Match' object has no attribute 'getitem'

[JimmXinu]

It's an issue with py2 and py3 re being slightly different.

Test PI posted.

[gille90]

fixed thanks