search

crocodilestick / Calibre-Web-Automated

Calibre-Web but automated and with Calibre features! Fully automate and simplify your eBook set up!
https://github.com/crocodilestick/Calibre-Web-Automated
GNU General Public License v3.0
707 stars 15 forks source link

No custom covers on Kobo - Error in cover-enforcer.py #48

Closed rursache closed 1 day ago

rursache commented 2 months ago

i just bought a kobo device and tried to use CWA to sync some books to it.

a book didn't had a cover and added it manually by uploading a jpg. i see it correctly in the dashboard

CWA

however when i sync to kobo the custom covers are not added:

kobo

only epub files with the cover in them are correctly synced, the ones with custom covers are not.

in docker logs i see this error:

2024-08-15T19:07:46.578942540Z Traceback (most recent call last):
2024-08-15T19:07:46.578971261Z   File "/etc/calibre-web-automator/cover-enforcer.py", line 209, in <module>
2024-08-15T19:07:46.578983697Z     main()
2024-08-15T19:07:46.578986394Z   File "/etc/calibre-web-automator/cover-enforcer.py", line 189, in main
2024-08-15T19:07:46.578989288Z     enforcer.check_for_other_logs()
2024-08-15T19:07:46.578991406Z   File "/etc/calibre-web-automator/cover-enforcer.py", line 143, in check_for_other_logs
2024-08-15T19:07:46.579003534Z     book_info = self.enforce_cover(book_dir)
2024-08-15T19:07:46.579014733Z   File "/etc/calibre-web-automator/cover-enforcer.py", line 65, in enforce_cover
2024-08-15T19:07:46.579020525Z     epub = [f for f in library_files if f.endswith('.epub')][0]
2024-08-15T19:07:46.579030599Z IndexError: list index out of range

my docker run command:

docker run -d \
  --name cwa \
  --restart unless-stopped \
  -p 8786:8083 \
  -e PUID=1000 \
  -e PGID=1000 \
  -e TZ=Europe/Bucharest \
  -v ~/.calibre-web/cristina/config:/config \
  -v ~/.calibre-web/cristina/import:/cwa-book-ingest \
  -v ~/.calibre-web/cristina/library:/calibre-main \
  crocodilestick/calibre-web-automated:latest

the container is at https://books.domain.tld. server proxy port is set to 80 for the kobo sync to work

crocodilestick commented 2 months ago

Hey @rursache, could you please post the paths for the epubs in the library?

E.g. /calibre-main/Colleen Hoover/It Ends With Us (21)/It Ends With Us - Colleen Hoover.epub

I suspect it's a problem that's been fixed in the RC version with none alphanumeric characters in titles and author names but I just want to make sure.

rursache commented 2 months ago

@crocodilestick

~/.calibre-web/cristina/library (in docker that would be /calibre-main):

├── Agatha Christie
│   ├── Cand Ma Furnica Degetele (23)
│   │   ├── Cand Ma Furnica Degetele - Agatha Christie.epub
│   │   ├── cover.jpg
│   │   └── metadata.opf
│   └── E Usor Sa Ucizi (22)
│       ├── cover.jpg
│       ├── E Usor Sa Ucizi - Agatha Christie.epub
│       └── metadata.opf
├── Alice Oseman
│   └── Loveless (25)
│       ├── cover.jpg
│       ├── Loveless - Alice Oseman.epub
│       └── metadata.opf
├── Andrei Ciobanu
│   └── Suge-o, Ramona! (19)
│       ├── cover.jpg
│       ├── metadata.opf
│       └── Suge-o, Ramona! - Andrei Ciobanu.epub
├── Colleen Hoover
│   └── It Ends With Us (21)
│       ├── cover.jpg
│       ├── It Ends With Us - Colleen Hoover.epub
│       └── metadata.opf
├── David Graeber
│   └── Bullshit Jobs (20)
│       ├── Bullshit Jobs - David Graeber.epub
│       ├── cover.jpg
│       └── metadata.opf
├── Elizabeth _Eliza_ Montgomery
│   └── Love's Timeless Voyage (26)
│       ├── cover.jpg
│       ├── Love's Timeless Voyage - Elizabeth _Eliza_ Montgomery.epub
│       └── metadata.opf
├── Ilsa Madden-Mills
│   └── Dear Ava (24)
│       ├── cover.jpg
│       ├── Dear Ava - Ilsa Madden-Mills.epub
│       └── metadata.opf
├── metadata.db
└── Shwetabh Gangwar
    └── The Rudest Book Ever (27)
        ├── cover.jpg
        ├── metadata.opf
        └── The Rudest Book Ever - Shwetabh Gangwar.epub

18 directories, 28 files

i "fixed" the issue by extracting the 3 epubs causing issues in a separate folder, removing them (from book list) in CWA and the re-importing them. they got reimported correctly and synced to kobo as expected. also no more errors in docker logs

this "fix" is quite annoying tho. i'm trying to setup this for my girlfriend and i can't ask her to do all this everytime she updates the metadata of a file

unrelated, i would also love a way to upload a non-epub file through the dashboard instead of using the /cwa-book-ingest folder. something like "upload and convert" next to the already existing "upload"

jmarmstrong1207 commented 2 months ago

Could you try seeing if the new version works for you?

jmarmstrong1207 commented 2 months ago

It looks like the code isn't double checking if the array epub = [f for f in library_files if f.endswith('.epub')][0] is not null before checking index [0]. Should be fixed if the code double checks that.

crocodilestick commented 1 month ago

It looks like the code isn't double checking if the array epub = [f for f in library_files if f.endswith('.epub')][0] is not null before checking index [0]. Should be fixed if the code double checks that.

Wouldn't really help anything though, I can add a sys.exit(code) so the error looks nicer but at the end of the day if that list is empty there are issues elsewhere

crocodilestick commented 1 month ago

It looks like the code isn't double checking if the array epub = [f for f in library_files if f.endswith('.epub')][0] is not null before checking index [0]. Should be fixed if the code double checks that.

Wouldn't really help anything though, I can add a sys.exit(code) so the error looks nicer but at the end of the day if that list is empty there are issues elsewhere.

Out of the 5 books tested above, the 2 that functioned normally are the ones without any special characters in the book title or author name. I had a very similar issue with the Stephen King book 11/22/63 because of the forward slashes in the title which were automatically replaced with _s by Calibre and hence the expected path of the edited ebook was wrong.

That's been fixed but I may need to investigate more with more characters.

jmarmstrong1207 commented 1 month ago

There's a for loop that calls that function, so if it errors out, it will just exit entirely and not finish the for loop, potentially causing cover enforcer to not go through all books, right?

crocodilestick commented 1 month ago

There's a for loop that calls that function, so if it errors out, it will just exit entirely and not finish the for loop, potentially causing cover enforcer to not go through all books, right?

Only if cover-enforcer -all was manually ran, changes in the Web UI are processed one by one.

Added error exceptions to cover enforcer for the upcoming versions but also still looking for documentation on the filename character substitutions Calibre makes during import as it can be confusing.

E.g. it conforms to windows filename limitations e.g. no *<>|"/\?: but also no .. but sometimes .. is allowed. There also doesn't seem to be a set length at which a filename will be truncated, and it's support for characters from other languages e.g. äßãáæ is inconsistent at best

jmarmstrong1207 commented 1 month ago

fyi, it looks like this is also an issue in stock calibre-web. Can't modify kobo covers. https://github.com/janeczku/calibre-web/issues/2509

Also, I can reproduce a similar error you have and these errors are likely unrelated to the kobo over issue.

    return [f for f in temp_files if f.endswith('.opf')][0]
IndexError: list index out of range
crocodilestick commented 1 month ago

@jmarmstrong1207 seems to have solved it, it seems as though Kobo is responsible for caching a books data & cover and it's not super responsive, see below for a work around:

https://github.com/janeczku/calibre-web/issues/2509#issuecomment-1607815394

jmarmstrong1207 commented 1 month ago

Good news: There is a single-line patch apparently that can help automatically update the metadata to kobo devices. https://github.com/janeczku/calibre-web/issues/2816#issuecomment-2309026841. If stock calibre-web doesn't patch it (unlikely soon), then we can do it. I'm not sure if covers count as metadata

crocodilestick commented 1 month ago

Good news: There is a single-line patch apparently that can help automatically update the metadata to kobo devices. janeczku/calibre-web#2816 (comment). If stock calibre-web doesn't patch it (unlikely soon), then we can do it. I'm not sure if covers count as metadata

Amazing, adding @tsheinen's patch to V2.1.0 regardless of whether stock CW adds it in time 👍

jmarmstrong1207 commented 1 month ago

Apparently this patch doesn't update the cover, only other metadata changes, so this issue should remain open

tsheinen commented 1 month ago

FYI: https://github.com/tsheinen/calibre-web/commit/9dd16f848de143ba68b1cf9702ed3b4b6ae57896 does update the cover although i'm still workshopping exactly how the "cover-specific UUID" backend works and I haven't decided how best to invalidate sync records to actually trigger the re-sync.

Probably wouldn't recommend actually applying that patch unless you really want that capability now since database things might change -- but I have all the components needed just need to actually put it together in a good way.

crocodilestick commented 1 day ago

Issue fixed in Version 2.1.0