Fix this bug: Heroku deployment not working

[krispykiq]

I tried using the deploy from heroku button and filled out all the needed envinroment variables.

It said:

-----> Build succeeded!
-----> Python app detected
 !     Requested runtime (python-3.6.9) is not available for this stack (heroku-20).
 !     Aborting.  More info: https://devcenter.heroku.com/articles/python-support
 !     Push rejected, failed to compile Python app.
 !     Push failed

So I tried fixing it by cloning the repository and changing the python version in runtime.txt to python-3.6.13 but then another error appeared saying

ar: libnss3_3.58-1_amd64.deb: file format not recognized
 !     Push rejected, failed to compile Calibre app.
 !     Push failed

[damare01]

please fix it!

[dipu-bd]

Have you tried using python 3.8?

[damare01]

Have you tried using python 3.8?

won't work

[damare01]

Have you tried using python 3.8?

won't work

this is what I get

ar: libnss3_3.58-1_amd64.deb: file format not recognized
 !     Push rejected, failed to compile Calibre app.
 !     Push failed

[damare01]

tried this versions python-3.9.4 python-3.8.9 python-3.7.10 python-3.6.13 it doesn't work. I always get this

ar: libnss3_3.58-1_amd64.deb: file format not recognized
 !     Push rejected, failed to compile Calibre app.
 !     Push failed

[krispykiq]

I'm not really sure about this but is the problem the heroku calibre buildpack that's being used? Is it outdated now or something?

[krispykiq]

I managed to make it work by changing

http://ftp.debian.org/debian/pool/main/n/nss/libnss3_3.58-1_amd64.deb

to

http://ftp.debian.org/debian/pool/main/n/nss/libnss3_3.61-1_amd64.deb

in heroku-buildpack-calibre/bin/compile

that will fix the deployment problem but the discord bot I made was offline so I turned on the dyno

bot python . --shard-id 0 --shard-count 1

and that will make it online

It's pretty much alright after I did that, although the book format "pdf" doesn't work. Every book format except pdf works alright. I tried the discord bot in your discord server and the pdf format works so I really don't know what's the problem. Do you know what's causing this?

2021-04-22T13:32:38.746169+00:00 app[bot.1]:   File "site.py", line 45, in <module>
2021-04-22T13:32:38.746169+00:00 app[bot.1]:   File "site.py", line 41, in main
2021-04-22T13:32:38.746169+00:00 app[bot.1]:   File "calibre/ebooks/conversion/cli.py", line 401, in main
2021-04-22T13:32:38.746170+00:00 app[bot.1]:   File "calibre/ebooks/conversion/plumber.py", line 1103, in run
2021-04-22T13:32:38.746171+00:00 app[bot.1]:   File "calibre/ebooks/conversion/plugins/pdf_output.py", line 151, in specialize_options
2021-04-22T13:32:38.746172+00:00 app[bot.1]: ImportError: cannot import name 'QWebEngineUrlScheme' from 'qt.webengine' (/app/calibre/lib/calibre-extensions/python-lib.bypy.frozen/qt/webengine.pyc)
2021-04-22T13:32:43.282659+00:00 app[bot.1]: 13:32:43 INFO     Compressing output...
2021-04-22T13:32:43.283068+00:00 app[bot.1]: 13:32:43 INFO     Not archiving single file inside /app/.discord_bot_output/736190884516724738/My Observational Diary of Elena Evoy/epub
2021-04-22T13:32:43.283295+00:00 app[bot.1]: 13:32:43 INFO     It has no files: /app/.discord_bot_output/736190884516724738/My Observational Diary of Elena Evoy/pdf
2021-04-22T13:32:48.722171+00:00 app[bot.1]: 13:32:48 INFO     Destroyed App

these are the logs

[dipu-bd]

pdf is generated using calibre. maybe calibre is not installed properly. I can see there are some errors regarding calibre in your output log

[dipu-bd]

@NNTin can you take a look at this issue?

[NNTin]

Should be easy to fix but I don't know when I'll find the time. May take a few days.

[dipu-bd]

That's good. Please fix it when you are free. Also, please change the python version to 3.8 if it not too much work.

[damare01]

heroku-buildpack-calibre/bin/compile

hello @krispykiq where can I fid this directory?

[NNTin]

I can't get around the slug size problem. I assume for pdf support python3-pyqt5.qtwebengine needs to be added to Aptfile or PyQtWebEngine needs to be added to requirements.txt to fix the error ImportError: cannot import name 'QWebEngineUrlScheme' from 'qt.webengine'

The webengine is based on Chromium browser and is quite large.

However when doing so the slug size becomes too big >500 MB leading to the app not being deployed at all.

Current slug size is 386 MB. Adding any of those dependencies adds at least 129 MB on top.

I did look into clearing the cache but they have no effect on the slug size. (Running rm -rf $CACHE_DIR at the verry end.)

Docker images is not affected by slug size but I don't know how to integrate that with the Heroku Deploy button.

Here calibre switched from PyQt5.QtWebEngineCore.QWebEngineUrlScheme to qt.webengine.QWebEngineUrlScheme.

I tried making sense what exactly changed and why we can no longer find qt.webengine.QWebEngineUrlScheme

The change was done due to: http://python.6.x6.nabble.com/Various-Issues-with-PyQt6-td5287230.html

Fortunately, this too is trivial to fix thanks to the ability to override getattr for modules in python 3.7+. Now in calibre, I can just import any PyQt5 thing using

from qt.core import whatever from qt.webengine import whatever

And if I decide to switch to PyQt6 or PySide or anything else, it's a one line change. And I dont need to keep track of which module contains which class. Or have multiple import statements for PyQt. And thanks to auto generated .pyi files, autocompletion even works with jedi.

If anyone is interested, here is the wrapper code to make it happen https://github.com/kovidgoyal/calibre/tree/master/src/qt

Just copy that directory onto your PYTHONPATH. You can regenerate/update it by simply running python /path/to/qt/folder. Look inside main.py for the list of Qt modules that are placed into each namespace. The list is suited for calibre, adjust to your needs.

I'm wondering why in the past the PDF generation worked. As far as I can tell we didn't install PyQt (dependency not listed in Aptfile or requirements.txt) and the Calibre installer script didn't install it either (I deployed a fresh bot in November 2020 and PDF was working.).

[NNTin]

Updated PR. PDF should now work.

[dipu-bd]

I'm wondering why in the past the PDF generation worked. As far as I can tell we didn't install PyQt (dependency not listed in Aptfile or requirements.txt) and the Calibre installer script didn't install it either (I deployed a fresh bot in November 2020 and PDF was working.).

For pdf generation, I used only the ebook-convert bin in calibre. Maybe it does not depend on PyQT

[dipu-bd]

However when doing so the slug size becomes too big >500 MB leading to the app not being deployed at all.

Slug size is really causing a problem. It would be great if we could use only the ebook-convert bin without calibre. I will look into it if there is any way.