search

xxyzz / WordDumb

A calibre plugin that generates Kindle Word Wise and X-Ray files for KFX, AZW3, MOBI and EPUB eBook.
https://xxyzz.github.io/WordDumb/
GNU General Public License v3.0
376 stars 19 forks source link

i get an error when generating wordwise and xray #231

Closed daenghunter closed 1 month ago

daenghunter commented 3 months ago

Checkboxes

Describe the bug and copy the error message

calibre, version 7.12.0 (win32, embedded-python: True) Tonnerre de Brest!: An error occurred, please copy error message then report bug at GitHub.

Starting job: Generating Word Wise and X-Ray for A Psalm for the Wild-Built Job: "Generating Word Wise and X-Ray for A Psalm for the Wild-Built" failed with error: Traceback (most recent call last): File "calibre\gui2\threaded_jobs.py", line 85, in start_work File "calibre_plugins.worddumb.parse_job", line 211, in do_job File "calibre_plugins.worddumb.utils", line 53, in run_subprocess File "subprocess.py", line 571, in run subprocess.CalledProcessError: Command '['py', 'C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', '{"book_id": 3, "book_path": "C:\\Users\\OWNER\\OneDrive\\Calibre\\Becky Chambers\\A Psalm for the Wild-Built (3)\\A Psalm for the Wild-Built - Becky Chambers.kfx", "mi": null, "book_fmt": "KFX", "book_lang": "en", "useragent": "WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)", "plugin_path": "C:\\Users\\OWNER\\AppData\\Roaming\\calibre\\plugins\\WordDumb.zip", "spacy_model": "en_core_web_md", "create_ww": true, "create_x": true, "asin": "B08H831J18", "acr": "CR!YPSVDULJ5J3O5INDW2BHCZ8GEBKN", "revision": "", "kfx_json": null, "mobi_html": null, "mobi_codec": ""}', '{"use_pos": true, "search_people": false, "model_size": "md", "zh_wiki_variant": "cn", "mediawiki_api": "", "add_locator_map": false, "preferred_formats": ["KFX", "AZW3", "AZW", "MOBI", "EPUB"], "use_all_formats": false, "minimal_x_ray_count": 1, "choose_format_manually": false, "wiktionary_gloss_lang": "en", "kindle_gloss_lang": "en", "use_gpu": false, "cuda": "cu121", "use_wiktionary_for_kindle": false, "remove_link_styles": false, "python_path": "", "show_change_kindle_ww_lang_warning": true, "ca_wiktionary_difficulty_limit": 5, "cs_wiktionary_difficulty_limit": 5, "da_wiktionary_difficulty_limit": 5, "de_wiktionary_difficulty_limit": 5, "el_wiktionary_difficulty_limit": 5, "en_wiktionary_difficulty_limit": 5, "es_wiktionary_difficulty_limit": 5, "fi_wiktionary_difficulty_limit": 5, "fr_wiktionary_difficulty_limit": 5, "he_wiktionary_difficulty_limit": 5, "hr_wiktionary_difficulty_limit": 5, "it_wiktionary_difficulty_limit": 5, "ja_wiktionary_difficulty_limit": 5, "ko_wiktionary_difficulty_limit": 5, "lt_wiktionary_difficulty_limit": 5, "mk_wiktionary_difficulty_limit": 5, "nl_wiktionary_difficulty_limit": 5, "no_wiktionary_difficulty_limit": 5, "pl_wiktionary_difficulty_limit": 5, "pt_wiktionary_difficulty_limit": 5, "ro_wiktionary_difficulty_limit": 5, "ru_wiktionary_difficulty_limit": 5, "sl_wiktionary_difficulty_limit": 5, "sv_wiktionary_difficulty_limit": 5, "uk_wiktionary_difficulty_limit": 5, "zh_wiktionary_difficulty_limit": 5}']' returned non-zero exit status 1.

Called with args: (ParseJobData(book_id=3, book_path='C:\Users\OWNER\OneDrive\Calibre\Becky Chambers\A Psalm for the Wild-Built (3)\A Psalm for the Wild-Built - Becky Chambers.kfx', mi=<calibre.ebooks.metadata.book.base.Metadata object at 0x000001AC05D2C610>, book_fmt='KFX', book_lang='en', useragent='WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)', plugin_path='C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', spacy_model='en_core_web_md', create_ww=True, create_x=True, asin='B08H831J18', acr='CR!YPSVDULJ5J3O5INDW2BHCZ8GEBKN', revision='', kfx_json=None, mobi_html=None, mobi_codec=''),) {'notifications': <queue.Queue object at 0x000001AC09001A10>, 'abort': <threading.Event at 0x1ac09003890: unset>, 'log': <calibre.utils.logging.GUILog object at 0x000001AC09003390>} Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip__main.py", line 37, in File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 237, in create_files File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 763, in load_spacy File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\init.py", line 6, in from .errors import setup_default_warnings File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\errors.py", line 3, in from .compat import Literal File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\compat.py", line 39, in from thinc.api import Optimizer # noqa: F401 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\api.py", line 1, in from .backends import ( File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends\init__.py", line 17, in from .cupy_ops import CupyOps File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends\cupy_ops.py", line 16, in from .numpy_ops import NumpyOps File "thinc\backends\numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Plugin settings and reproduce steps

so whenever i try to get the files for kfx files. it dpesnt work and i get the same error message, it happens in both of my laptops

even reinstalling and deleting all the plugins doesnt solve the problem

Generated files, screenshots or videos

image

xxyzz commented 3 months ago

Please delete "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\", some dependency packages probably are outdated.

xxyzz commented 3 months ago

New numpy 2.0 release breaks spaCy model... Downgrade numpy should fix the error.

daenghunter commented 3 months ago

i downgraded it but i still get an error

Starting job: Generating Word Wise and X-Ray for The Midnight Library Job: "Generating Word Wise and X-Ray for The Midnight Library" failed with error: Traceback (most recent call last): File "calibre\gui2\threaded_jobs.py", line 85, in start_work File "calibre_plugins.worddumb.parse_job", line 211, in do_job File "calibre_plugins.worddumb.utils", line 53, in run_subprocess File "subprocess.py", line 571, in run subprocess.CalledProcessError: Command '['py', 'C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', '{"book_id": 982, "book_path": "I:\\Calibre\\Matt. Haig\\The Midnight Library (982)\\The Midnight Library - Matt. Haig.kfx", "mi": null, "book_fmt": "KFX", "book_lang": "en", "useragent": "WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)", "plugin_path": "C:\\Users\\OWNER\\AppData\\Roaming\\calibre\\plugins\\WordDumb.zip", "spacy_model": "en_core_web_md", "create_ww": true, "create_x": true, "asin": "BB92627JZS", "acr": "CR!D4IWF42E8MMGJAX9GJCLKCZCTK5V", "revision": "", "kfx_json": null, "mobi_html": null, "mobi_codec": ""}', '{"use_pos": true, "search_people": true, "model_size": "md", "zh_wiki_variant": "cn", "mediawiki_api": "", "add_locator_map": false, "preferred_formats": ["KFX", "AZW3", "AZW", "MOBI", "EPUB"], "use_all_formats": false, "minimal_x_ray_count": 1, "choose_format_manually": false, "wiktionary_gloss_lang": "en", "kindle_gloss_lang": "en", "use_gpu": false, "cuda": "cu121", "use_wiktionary_for_kindle": false, "remove_link_styles": false, "python_path": "", "show_change_kindle_ww_lang_warning": true, "ca_wiktionary_difficulty_limit": 5, "cs_wiktionary_difficulty_limit": 5, "da_wiktionary_difficulty_limit": 5, "de_wiktionary_difficulty_limit": 5, "el_wiktionary_difficulty_limit": 5, "en_wiktionary_difficulty_limit": 5, "es_wiktionary_difficulty_limit": 5, "fi_wiktionary_difficulty_limit": 5, "fr_wiktionary_difficulty_limit": 5, "he_wiktionary_difficulty_limit": 5, "hr_wiktionary_difficulty_limit": 5, "it_wiktionary_difficulty_limit": 5, "ja_wiktionary_difficulty_limit": 5, "ko_wiktionary_difficulty_limit": 5, "lt_wiktionary_difficulty_limit": 5, "mk_wiktionary_difficulty_limit": 5, "nl_wiktionary_difficulty_limit": 5, "no_wiktionary_difficulty_limit": 5, "pl_wiktionary_difficulty_limit": 5, "pt_wiktionary_difficulty_limit": 5, "ro_wiktionary_difficulty_limit": 5, "ru_wiktionary_difficulty_limit": 5, "sl_wiktionary_difficulty_limit": 5, "sv_wiktionary_difficulty_limit": 5, "uk_wiktionary_difficulty_limit": 5, "zh_wiktionary_difficulty_limit": 5}']' returned non-zero exit status 1.

Called with args: (ParseJobData(book_id=982, book_path='I:\Calibre\Matt. Haig\The Midnight Library (982)\The Midnight Library - Matt. Haig.kfx', mi=<calibre.ebooks.metadata.book.base.Metadata object at 0x0000018CC97B3190>, book_fmt='KFX', book_lang='en', useragent='WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)', plugin_path='C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', spacy_model='en_core_web_md', create_ww=True, create_x=True, asin='BB92627JZS', acr='CR!D4IWF42E8MMGJAX9GJCLKCZCTK5V', revision='', kfx_json=None, mobi_html=None, mobi_codec=''),) {'notifications': <queue.Queue object at 0x0000018CD0FC7E50>, 'abort': <threading.Event at 0x18cc97b1e50: unset>, 'log': <calibre.utils.logging.GUILog object at 0x0000018CC97B1710>} Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip__main.py", line 37, in File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 237, in create_files File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 763, in load_spacy File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\init.py", line 6, in from .errors import setup_default_warnings File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\errors.py", line 3, in from .compat import Literal File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\compat.py", line 39, in from thinc.api import Optimizer # noqa: F401 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\api.py", line 1, in from .backends import ( File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends\init__.py", line 17, in from .cupy_ops import CupyOps File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends\cupy_ops.py", line 16, in from .numpy_ops import NumpyOps File "thinc\backends\numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

hosford42 commented 3 months ago

I'm getting this too. Freshly installed SpaCy, with no other packages installed. I don't use your package at all, @xxyzz , so I'm thinking this is a SpaCy issue.

hosford42 commented 3 months ago

Confirmed:

https://github.com/explosion/spaCy/issues/13528

xxyzz commented 3 months ago

This plugin downloads its dependencies in calibre's plugin folder and doesn't use packages installed for the system. A temporary fix is downgrade numpy in that folder manually:

py -m pip install -U -t C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12 numpy==1.26.4

Please notice the install path is for Python 3.12, it needs to be changed for other Python versions.

woofingcough commented 3 months ago

This plugin downloads its dependencies in calibre's plugin folder and doesn't use packages installed for the system. A temporary fix is downgrade numpy in that folder manually:

py -m pip install -U -t C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12 numpy==1.16.4

Could you give me a command for MacOS as well?

xxyzz commented 3 months ago

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.26.4
woofingcough commented 3 months ago

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.16.4

Much appreciated!

woofingcough commented 3 months ago

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.16.4

By the way, the problem persists even though I have applied this… and 1.16.4 has been deprecated, so I used 1.26.4.

u0398 commented 3 months ago

This plugin downloads its dependencies in calibre's plugin folder and doesn't use packages installed for the system. A temporary fix is downgrade numpy in that folder manually:

py -m pip install -U -t C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12 numpy==1.16.4

I could not get 1.16.4 to install (ERROR: Failed building wheel for numpy), but 1.26.4 installed and WordDumb is working.

xxyzz commented 3 months ago

Sorry for the typo🤦 Should downgrade to 1.26.4

woofingcough commented 3 months ago

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.16.4

By the way, the problem persists even though I have applied this… and 1.16.4 has been deprecated, so I used 1.26.4.

Deleting the plugin and the remaining files and reinstalling the plugin again before typing in this solved the problem!