search

linuxserver / docker-calibre-web

GNU General Public License v3.0
1.02k stars 145 forks source link

Ebook-convert fails after Calibre v6 release #197

Closed buurman closed 2 years ago

buurman commented 2 years ago

linuxserver.io

Expected Behavior

Ebook should convert to Mobi, or any other format

Current Behavior

The convert fails.

Steps to Reproduce

  1. Select epub
  2. Edit
  3. Convert from epub to mobi
  4. Push Convert button

Environment

OS: Synology DSM 7 and also Ubuntu 22.04 CPU architecture: x86_64

Command used to create docker container (run/create/compose/screenshot)

As explained on https://hub.docker.com/r/linuxserver/calibre-web

Docker logs

[2022-07-11 20:18:11,807] INFO {cps.editbooks:1146} converting: book id: 6830 from: EPUB to: MOBI [2022-07-11 20:18:11,823] DEBUG {cps.services.worker:88} Add Task for user: ### - Convert 6830 None [2022-07-11 20:18:11,834] INFO {cps.tasks.convert:142} Book id 6830 - target format of .mobi does not exist. Moving forward with convert. [2022-07-11 20:18:13,370] DEBUG {cps.tasks.convert:253} No write access to /root/.config/calibre using a temporary dir instead [2022-07-11 20:18:13,613] DEBUG {cps.tasks.convert:253} 1% Converting input to HTML... [2022-07-11 20:18:13,614] DEBUG {cps.tasks.convert:253} InputFormatPlugin: EPUB Input running [2022-07-11 20:18:13,615] DEBUG {cps.tasks.convert:253} on /books/###/###/###.epub [2022-07-11 20:18:14,080] DEBUG {cps.tasks.convert:253} Found HTML cover Ops/001.html [2022-07-11 20:18:14,125] DEBUG {cps.tasks.convert:253} Parsing all content... [2022-07-11 20:18:14,925] DEBUG {cps.tasks.convert:253} 34% Running transforms on e-book... [2022-07-11 20:18:15,595] DEBUG {cps.tasks.convert:253} Merging user specified metadata... [2022-07-11 20:18:15,597] DEBUG {cps.tasks.convert:253} Detecting structure... [2022-07-11 20:18:15,652] DEBUG {cps.tasks.convert:253} Detected chapter: PROLOGUE [2022-07-11 20:18:15,653] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER ONE [2022-07-11 20:18:15,655] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWO [2022-07-11 20:18:15,656] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THREE [2022-07-11 20:18:15,657] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER FOUR [2022-07-11 20:18:15,659] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER FIVE [2022-07-11 20:18:15,660] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER SIX [2022-07-11 20:18:15,661] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER SEVEN [2022-07-11 20:18:15,663] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER EIGHT [2022-07-11 20:18:15,664] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER NINE [2022-07-11 20:18:15,665] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TEN [2022-07-11 20:18:15,666] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER ELEVEN [2022-07-11 20:18:15,668] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWELVE [2022-07-11 20:18:15,669] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTEEN [2022-07-11 20:18:15,670] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER FOURTEEN [2022-07-11 20:18:15,672] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER FIFTEEN [2022-07-11 20:18:15,673] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER SEVENTEEN [2022-07-11 20:18:15,674] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER EIGHTEEN [2022-07-11 20:18:15,676] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER NINETEEN [2022-07-11 20:18:15,677] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY [2022-07-11 20:18:15,678] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY ONE [2022-07-11 20:18:15,679] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY TWO [2022-07-11 20:18:15,681] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY THREE [2022-07-11 20:18:15,682] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY FOUR [2022-07-11 20:18:15,683] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY FIVE [2022-07-11 20:18:15,685] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY SIX [2022-07-11 20:18:15,686] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY SEVEN [2022-07-11 20:18:15,687] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY EIGHT [2022-07-11 20:18:15,689] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER TWENTY NINE [2022-07-11 20:18:15,690] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY [2022-07-11 20:18:15,692] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY ONE [2022-07-11 20:18:15,693] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY TWO [2022-07-11 20:18:15,694] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY THREE [2022-07-11 20:18:15,695] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY FOUR [2022-07-11 20:18:15,697] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY FIVE [2022-07-11 20:18:15,698] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY SIX [2022-07-11 20:18:15,700] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY SEVEN [2022-07-11 20:18:15,701] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY EIGHT [2022-07-11 20:18:15,702] DEBUG {cps.tasks.convert:253} Detected chapter: CHAPTER THIRTY NINE [2022-07-11 20:18:15,829] DEBUG {cps.tasks.convert:253} Flattening CSS and remapping font sizes... [2022-07-11 20:18:18,434] DEBUG {cps.tasks.convert:253} Source base font size is 13.08000pt [2022-07-11 20:18:19,820] DEBUG {cps.tasks.convert:253} Removing fake margins... [2022-07-11 20:18:21,279] DEBUG {cps.tasks.convert:253} Cleaning up manifest... [2022-07-11 20:18:21,280] DEBUG {cps.tasks.convert:253} Trimming unused files from manifest... [2022-07-11 20:18:21,359] DEBUG {cps.tasks.convert:253} Trimming 'Ops/001.html' from manifest [2022-07-11 20:18:21,364] DEBUG {cps.tasks.convert:253} Creating MOBI Output... [2022-07-11 20:18:21,365] DEBUG {cps.tasks.convert:253} 67% Running MOBI Output plugin [2022-07-11 20:18:21,736] DEBUG {cps.tasks.convert:267} Failed to import PyQt module: PyQt6.QtCore with error: libQt6Core.so.6: cannot open shared object file: No such file or directory [2022-07-11 20:18:21,736] DEBUG {cps.tasks.convert:267} Traceback (most recent call last): [2022-07-11 20:18:21,737] DEBUG {cps.tasks.convert:267} File "runpy.py", line 196, in _run_module_as_main [2022-07-11 20:18:21,737] DEBUG {cps.tasks.convert:267} File "runpy.py", line 86, in _run_code [2022-07-11 20:18:21,738] DEBUG {cps.tasks.convert:267} File "site.py", line 45, in [2022-07-11 20:18:21,738] DEBUG {cps.tasks.convert:267} File "site.py", line 41, in main [2022-07-11 20:18:21,738] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/cli.py", line 419, in main [2022-07-11 20:18:21,739] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/plumber.py", line 1281, in run [2022-07-11 20:18:21,739] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/plugins/mobi_output.py", line 184, in convert [2022-07-11 20:18:21,739] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-11 20:18:21,740] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/mobi/writer2/resources.py", line 13, in [2022-07-11 20:18:21,740] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-11 20:18:21,741] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/mobi/utils.py", line 12, in [2022-07-11 20:18:21,741] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-11 20:18:21,741] DEBUG {cps.tasks.convert:267} File "calibre/utils/img.py", line 12, in [2022-07-11 20:18:21,742] DEBUG {cps.tasks.convert:267} ImportError: cannot import name 'QBuffer' from 'qt.core' (/app/calibre/lib/calibre-extensions/python-lib.bypy.frozen/qt/core.pyc) [2022-07-11 20:18:21,743] INFO {cps.tasks.convert:189} ebook converter failed with error while converting book

github-actions[bot] commented 2 years ago

Thanks for opening your first issue here! Be sure to follow the bug or feature issue templates!

stefore commented 2 years ago

Hi. I've got the same problem since the last update.

[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267} Failed to import PyQt module: PyQt6.QtGui with error: libEGL.so.1: cannot open shared object file: No such file or directory
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267} Traceback (most recent call last):
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "runpy.py", line 196, in _run_module_as_main
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "runpy.py", line 86, in _run_code
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "site.py", line 45, in <module>
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "site.py", line 41, in main
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "calibre/ebooks/conversion/cli.py", line 419, in main
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "calibre/ebooks/conversion/plumber.py", line 1281, in run
[2022-07-13 09:31:14,980] DEBUG {cps.tasks.convert:267}   File "calibre/ebooks/conversion/plugins/mobi_output.py", line 184, in convert
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267}   File "bypy-importer.py", line 279, in exec_module
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267}   File "calibre/ebooks/mobi/writer2/resources.py", line 13, in <module>
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267}   File "bypy-importer.py", line 279, in exec_module
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267}   File "calibre/ebooks/mobi/utils.py", line 12, in <module>
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267}   File "bypy-importer.py", line 279, in exec_module
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267}   File "calibre/utils/img.py", line 12, in <module>
[2022-07-13 09:31:14,981] DEBUG {cps.tasks.convert:267} ImportError: cannot import name 'QColor' from 'qt.core' (/app/calibre/lib/calibre-extensions/python-lib.bypy.frozen/qt/core.pyc)
[2022-07-13 09:31:14,981]  INFO {cps.tasks.convert:189} ebook converter failed with error while converting book
thespad commented 2 years ago

Can you both please provide the output of uname -a on your Docker hosts.

buurman commented 2 years ago

Linux 5ff8e2e4aa3f 3.10.108 #42661 SMP Mon Jun 27 15:03:33 CST 2022 x86_64 x86_64 x86_64 GNU/Linux

stefore commented 2 years ago

Linux 01f3b0456896 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11) x86_64 x86_64 x86_64 GNU/Linux

thespad commented 2 years ago

@buurman there's a known issue with Synology 3.x kernels and Calibre 6.x - we don't have a fix at the moment and may not do given the age of the kernel.

@stefore there's no specific known issues with 4.x kernels but yours is from 2019 so it's entirely likely that's the source of your issue.

stefore commented 2 years ago

thanks @thespad , but after :

i still have the same problem :(

thespad commented 2 years ago

Oh, I think I might know the issue, give me a bit to spin up a test.

buurman commented 2 years ago

I gave it another try on my NUC8 with Ubuntu 22.04 and get the exact same error as on my Synology.

uname -a output >> Linux 2cc69c211315 5.15.0-41-generic #44-Ubuntu SMP Wed Jun 22 14:20:53 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

I found a sort of similar error as above mentioned (Failed to import PyQt module: PyQt6.QtCore with error: libQt6Core.so.6: cannot open shared object file: No such file or directory) and the developer of Calibre responded with that Calibre is likely missing one of the needed system libraries. source

buurman commented 2 years ago

I think I fixed it for the Synology ageing kernel.. albeit a little bit of a dirty fix. This is what I found source

1) fix the v6 deps, as in the new commit from @thespad >> apt-get install libgl1-mesa-glx libxdamage1 libegl1 libxkbcommon0 libopengl0 2) remove the ABI-tag from the libQt6Core library which doesn't seem to allow the old kernel version, binutils is needed >> strip --remove-section=.note.ABI-tag /app/calibre/lib/libQt6Core.so.6

After that everything seems to work ok.

Mirror0 commented 2 years ago

I have the same problem with ImportError: cannot import name 'QColor' from 'qt.core' on docker deployed on a standard Ubuntu 22.04 server. I solved it by manual installation of missing packages (libopengl-dev, libxkbcommon-dev, libegl-dev) within the docker container. @thespad I'm not familiar with this libraries or dockerfiles is the fixes provided in the linked PR will be applicable also for general use image or is it only for Synology?

thespad commented 2 years ago

The PR will fix it across the board, the Synology 3.x kernel issue is separate and not something we can fix in the image without impacting other users.

FWIW you don't need the -dev packages, the regular ones are fine.

gromez commented 2 years ago

For those who are on Synology 3.x kernel like me, just force the last 5.x version: DOCKER_MODS=linuxserver/calibre-web:5.44.0-calibre

riddlecp commented 2 years ago

Experiencing the same issue - failure during ebook convert

host system info: Rockstor (OpenSuse Leap 15.3) Linux rockstor1 5.18.2-lp153.4.g6d13af9-default #1 SMP PREEMPT_DYNAMIC Tue Jun 7 07:50:32 UTC 2022 (6d13af9) x86_64 x86_64 x86_64 GNU/Linux

Debug log: [2022-07-14 22:20:04,450] DEBUG {cps.tasks.convert:253} Flattening CSS and remapping font sizes... [2022-07-14 22:20:05,601] DEBUG {cps.tasks.convert:253} Source base font size is 12.00000pt [2022-07-14 22:20:06,260] DEBUG {cps.tasks.convert:253} Removing fake margins... [2022-07-14 22:20:06,714] DEBUG {cps.tasks.convert:253} Cleaning up manifest... [2022-07-14 22:20:06,714] DEBUG {cps.tasks.convert:253} Trimming unused files from manifest... [2022-07-14 22:20:06,740] DEBUG {cps.tasks.convert:253} Trimming 'OEBPS/toc.ncx' from manifest [2022-07-14 22:20:06,742] DEBUG {cps.tasks.convert:253} Creating MOBI Output... [2022-07-14 22:20:06,742] DEBUG {cps.tasks.convert:253} 67% Running MOBI Output plugin [2022-07-14 22:20:07,244] DEBUG {cps.tasks.convert:267} Failed to import PyQt module: PyQt6.QtGui with error: libEGL.so.1: cannot open shared object file: No such file or directory [2022-07-14 22:20:07,244] DEBUG {cps.tasks.convert:267} Traceback (most recent call last): [2022-07-14 22:20:07,244] DEBUG {cps.tasks.convert:267} File "runpy.py", line 196, in _run_module_as_main [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "runpy.py", line 86, in _run_code [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "site.py", line 45, in [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "site.py", line 41, in main [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/cli.py", line 419, in main [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/plumber.py", line 1281, in run [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/plugins/mobi_output.py", line 184, in convert [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/mobi/writer2/resources.py", line 13, in [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/mobi/utils.py", line 12, in [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-14 22:20:07,246] DEBUG {cps.tasks.convert:267} File "calibre/utils/img.py", line 12, in [2022-07-14 22:20:07,246] DEBUG {cps.tasks.convert:267} ImportError: cannot import name 'QColor' from 'qt.core' (/app/calibre/lib/calibre-extensions/python-lib.bypy.frozen/qt/core.pyc) [2022-07-14 22:20:07,246] INFO {cps.tasks.convert:189} ebook converter failed with error while converting book

buurman commented 2 years ago

Experiencing the same issue - failure during ebook convert

host system info: Rockstor (OpenSuse Leap 15.3) Linux rockstor1 5.18.2-lp153.4.g6d13af9-default #1 SMP PREEMPT_DYNAMIC Tue Jun 7 07:50:32 UTC 2022 (6d13af9) x86_64 x86_64 x86_64 GNU/Linux

Debug log: [2022-07-14 22:20:04,450] DEBUG {cps.tasks.convert:253} Flattening CSS and remapping font sizes... [2022-07-14 22:20:05,601] DEBUG {cps.tasks.convert:253} Source base font size is 12.00000pt [2022-07-14 22:20:06,260] DEBUG {cps.tasks.convert:253} Removing fake margins... [2022-07-14 22:20:06,714] DEBUG {cps.tasks.convert:253} Cleaning up manifest... [2022-07-14 22:20:06,714] DEBUG {cps.tasks.convert:253} Trimming unused files from manifest... [2022-07-14 22:20:06,740] DEBUG {cps.tasks.convert:253} Trimming 'OEBPS/toc.ncx' from manifest [2022-07-14 22:20:06,742] DEBUG {cps.tasks.convert:253} Creating MOBI Output... [2022-07-14 22:20:06,742] DEBUG {cps.tasks.convert:253} 67% Running MOBI Output plugin [2022-07-14 22:20:07,244] DEBUG {cps.tasks.convert:267} Failed to import PyQt module: PyQt6.QtGui with error: libEGL.so.1: cannot open shared object file: No such file or directory [2022-07-14 22:20:07,244] DEBUG {cps.tasks.convert:267} Traceback (most recent call last): [2022-07-14 22:20:07,244] DEBUG {cps.tasks.convert:267} File "runpy.py", line 196, in _run_module_as_main [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "runpy.py", line 86, in _run_code [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "site.py", line 45, in [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "site.py", line 41, in main [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/cli.py", line 419, in main [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/plumber.py", line 1281, in run [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/conversion/plugins/mobi_output.py", line 184, in convert [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/mobi/writer2/resources.py", line 13, in [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "calibre/ebooks/mobi/utils.py", line 12, in [2022-07-14 22:20:07,245] DEBUG {cps.tasks.convert:267} File "bypy-importer.py", line 279, in exec_module [2022-07-14 22:20:07,246] DEBUG {cps.tasks.convert:267} File "calibre/utils/img.py", line 12, in [2022-07-14 22:20:07,246] DEBUG {cps.tasks.convert:267} ImportError: cannot import name 'QColor' from 'qt.core' (/app/calibre/lib/calibre-extensions/python-lib.bypy.frozen/qt/core.pyc) [2022-07-14 22:20:07,246] INFO {cps.tasks.convert:189} ebook converter failed with error while converting book

Do this inside your Calibre-Web Docker >> apt-get install libgl1-mesa-glx libxdamage1 libegl1 libxkbcommon0 libopengl0

This does the same as thespad his pr and fixes the missing stuff

thespad commented 2 years ago

New v6-capable mod is available here: https://github.com/linuxserver/docker-mods/tree/universal-calibre

We will update the existing one to be v6 capable as well but it will be the last proper update for it. It really should have been moved to main mods repo a long time ago but we just haven't got around to it so please update your config when you can.