celogeek / go-comic-converter

Convert CBZ/CBR/Dir into Epub for e-reader devices (Kindle Devices, ...)
MIT License
67 stars 9 forks source link

Odd page behaviour #21

Closed yoanhg421 closed 12 months ago

yoanhg421 commented 1 year ago

Hi. I'm using the comic-converter to generate epubs

If you open it in Calibre Book Previewer you get a weird stretch on odd pages that wasn't there before version 2.2, I Think.

See the attached Screenshots. Books look fine on Kindle and iBooks but has a weird page stretch on every other page if reading on calibre.


Options: 
    Profile                   : KV - Kindle Paperwhite 3/4/Voyage/Oasis - 1072x1448
    Format                    : jpeg
    Quality                   : 80
    Grayscale                 : true
    Grayscale Mode            : normal
    Crop                      : false
    AutoContrast              : false
    AutoRotate                : true
    AutoSplitDoublePage       : false
    NoBlankImage              : true
    Manga                     : true
    HasCover                  : true
    LimitMb                   : 500 Mb
    StripFirstDirectoryFromToc: true
    SortPathMode              : path=alphanum, file=alpha
    Foreground Color          : #000
    Background Color          : #FFF
    Resize                    : true
    Aspect Ratio              : 1:1.60
    Portrait Only             : false
    Title Page                : always

Screenshots show window resized.

Screenshot 2023-09-20 at 8 28 32 AM Screenshot 2023-09-20 at 8 28 38 AM Screenshot 2023-09-20 at 8 28 53 AM Screenshot 2023-09-20 at 8 29 04 AM
celogeek commented 1 year ago

Hi, Indeed, the HTML/CSS I use to have perfect rendering on Kindle, seems not perfect on other viewer. It's hard to have something that works everywhere.

And Calibre is specially bad at displaying complex css, with double page rendering and landscape feature.

Can you show your settings ? may be some of them cause this ?

I think you can use the portrait only mode, may be that help, can you try ?

celogeek commented 1 year ago

Also try to force an aspect ratio of 1:1.6 (kindle advice)

yoanhg421 commented 1 year ago

I can try the portrait only,
I updated the first comment to show the settings. I know having one fits all it's very hard. let me test and will update.

yoanhg421 commented 1 year ago

OK. So portrait_only does fixes the problem. It must be a viewer issue then. It works great on everything else, and it's not like I read on Calibre anyways, but do verify the conversion went well before transferring the files. Is that an option for devices that support rotation? I'll try it in my scribe and see how it works.

yoanhg421 commented 1 year ago

Also. I wanted to ask you. My manga is usually in PNGs, and the folder size is usually smaller than the epub output, that's why I have the KV profile and quality set to 80. To save space.

I'm trying to find a setting that allows for smaller or same size epub output, or some setting that doesn't process the images too much maybe.

yoanhg421 commented 1 year ago
Screenshot 2023-09-20 at 9 25 06 AM Screenshot 2023-09-20 at 9 25 02 AM
celogeek commented 1 year ago

I'm using the profil SR (1200x1900). This is the maximum size amazon allow. In that specific case, converting ePub to kindle device mobi is very fast as amazon doesn't resize again the file. Also this one will work on any kindle device (smaller or bigger)

My quality is set to 90.

Usually the PNG files should be bigger, as it is a lossless compression of images compared to JPG. That's strange your files size is smaller with this format than the ePub resulting.

The Mobi file is always bigger than the ePub.

I do not convert locally, but use the web site sendtomykindle, which will convert and transfer. I also save the file resulting, in the amazon cloud, so I don't need to have a local copy on my drive or on my kindle device. When I want to read it, then I download it on my kindle, and when I have finish I remove the download from the kindle but keep it in the cloud.

The size limit on the website for the ePub is 200mb.

I only keep the original, in case I want to reencode with other settings.

Go Comic Converter

Options:
    Profile                   : SR - Standard Resolution - 1200x1920
    Format                    : jpeg
    Quality                   : 90
    Grayscale                 : true
    Grayscale Mode            : normal
    Crop                      : true
    CropRatio                 : 1 Left - 1 Up - 1 Right - 3 Bottom
    AutoContrast              : false
    AutoRotate                : false
    AutoSplitDoublePage       : false
    NoBlankImage              : true
    Manga                     : true
    HasCover                  : true
    LimitMb                   : 200 Mb
    StripFirstDirectoryFromToc: false
    SortPathMode              : path=alphanum, file=alpha
    Foreground Color          : #000
    Background Color          : #FFF
    Resize                    : true
    Aspect Ratio              : 1:1.60
    Portrait Only             : false
    Title Page                : always
celogeek commented 1 year ago

About the issue with calibre, I'm putting specific html and pagination parameter for supporting both Portrait and Landscape. For instance I have blank page that disappear on portrait and are only there on landscape for proper display of spread page. This feature is a kindle only feature, and doesn't work properly on other viewer that doesn't support it. Also I use the last version of ePub format, which is not always properly supported by other viewer.

The portrait mode, doesn't have additional settings for page spreading, that's why it work better on other devices.

yoanhg421 commented 1 year ago

That makes sense.

I’ll mess around with the settings. And see so far I get a decent quality on my Kindles and saves space.

You are right. The mobi files are always bigger. But they were too big. Total size of folder was 7.2Gb and total mobi output was like 9.4GB spread over 20 files with the 500MB limit. Now I get about the same fila size with 14 files.

I use the local converter for speed and library management with Calibre. Although Amazon decided to break the usb connection on the Scribe with the latest update. I was using the cloud one for a while but it used to take a while to upload convert and then download. Maybe now it will be faster that doesn’t have to reconvert the books with the new settings.

** By the way. I noticed the cover page doesn’t get processed like the rest. Is that intentional? It has a different aspect ratio from the rest of the book and has the grayscale banding issue you fixed months ago. The rest of the book looks fine.

Ok m not with my kindle right now but I can send you an image in about an hour to show you what I’m talking about.

celogeek commented 1 year ago

The cover has a different grayScale management yes. All the images use gray 16 bits, which support somethings like 4096 shade of grey. It is visible on the kindle, for some images, when 16 color trunk too much.

The problem is that the cover display, with the size it shows, have a very bad rendering in the listing if we are using the 4096 color shading. Amazon doesn't reconvert it to have proper render, so it looks bad.

So only for this one I use 16 shade of gray color, to improve the rendering of those thumbnails.

The counter part is that opening the cover full screen show a visible difference with the Title page (which is the cover + title in 4096 shade of gray).

But usually it's rare we open directly the cover.

yoanhg421 commented 1 year ago

Yes. It's rare to see it, most books open on the title page or the 1st page. You only see it if you navigate backwards from the title. I was about to test the lock screen. When you turn ON the setting to show the book cover when the screen is OFF. I wonder how it looks there.

yoanhg421 commented 1 year ago

It says it doesn't have a cover. But that's converting with kindlegen. That could be part of the problem. maybe the cloud convert shows it fine.

Thank for all your help. 👍

celogeek commented 1 year ago

When you transfering directly mobi file to the kindle, you also need to generate the thumbnail for the cover yourself. It's placed into an hidden directory on the device with the uuid of the book.

If you are using calibre, I think it got an option to do that for you.

If you are using the cloud, it will do it automatically.

celogeek commented 1 year ago

I recheck on calibre, what's wrong here. I've got terrible rendering on small size. It's better when I put it full screen, but calibre doesn't support 2 page at the same time, so the rendering looks bad as I alternate between right and left alignment. The end goal is to have both page as close as possible in landscape mode.

So I endup thinking that calibre doesn't show well right alignment. And usually you better go with kindle previewer or another better tools. (I use Apple Book for quick check, but it's not perfect here).

yoanhg421 commented 1 year ago

Yes. Calibre usually does the the Cover thumbnail for you that’s I found it strange that the cover wasn’t showing. Not sure what happened.

Yes. The double pages is terrible on small and large size. But it works well on Apple Books. I haven’t tried the double page anywhere else since I only have iPads a kindles. If I get another device I’ll give it a try.

manueldidonna commented 1 year ago

I have a problem that may be related to yours.

Screenshot 2023-09-23 alle 12 47 38

When I open the converted EPUB in Apple Books I get an extra page (Page 2 in the screenshot) that doesn't exist in the file. I used -titlepage=0 -hascover=0 options.

If I enable hascover I solve that problem but I get a different one: Screenshot 2023-09-23 alle 12 55 21 The cover is excluded from the pages and Apple Books treats the first page as the cover so it completely alters the layout by messing up double pages

yoanhg421 commented 12 months ago

Yes. So there are some weird black pages happening

There is also a blank page at the end of the book


    Author                    : GO Comic Converter
    Title                     : Chapter 1
    Workers                   : 10
    Profile                   : KV - Kindle Paperwhite 3/4/Voyage/Oasis - 1072x1448
    Format                    : jpeg
    Quality                   : 80
    Grayscale                 : true
    Grayscale Mode            : normal
    Crop                      : true
    CropRatio                 : 1 Left - 1 Up - 1 Right - 3 Bottom
    AutoContrast              : false
    AutoRotate                : false
    AutoSplitDoublePage       : true
    KeepDoublePageIfSplitted  : false
    NoBlankImage              : true
    Manga                     : true
    HasCover                  : true
    LimitMb                   : 500 Mb
    StripFirstDirectoryFromToc: true
    SortPathMode              : path=alphanum, file=alpha
    Foreground Color          : #000
    Background Color          : #FFF
    Resize                    : true
    Aspect Ratio              : 1:1.60
    Portrait Only             : false
    Title Page                : always

Thumbnails Overview

Screenshot 2023-09-23 at 5 17 23 AM

When Opening the book on page 1

Screenshot 2023-09-23 at 5 17 29 AM

Where the original double page used to be

Screenshot 2023-09-23 at 5 17 38 AM
celogeek commented 12 months ago

Hi,

Apple Books doesn't respect the rendition page spread settings I put on each page. And doesn't spread a double page on both side (so take left and right page to display a double page)

For manga mode, I start like this:

Then if I see a double page:

The issue is that Apple eBooks entirely ignore this settings. It will use an automatic mode and do the spread itself. And it doesn't support spreading a double page in landing mode. It mean instead we should remove the double page and use a splitted double page instead.

So, I can create a dedicated options to support Apple Books. This will do: (for manga mode)

It mean if you have a Kindle Scribe like me, and you don't want to split double page as you can see it perfectly well with the double page in full screen, that mode will be a bit less great. But fortunately, this mode will be supported by the kindle as the spread is properly supported there.

The only part the kindle will do, is to center the title page instead of spreading it to the left. A kindle seems to not be able to start on the left (center or right, that's why I put a blank page to start on the left).

So I will add an option like '--apple-book-compatibility' This way it will work fine there.

celogeek commented 12 months ago

Also by doing this, Apple Book will display properly the alignment I've done (right / left) on the images. For now it's weird because it put the image at the wrong place and align in the opposite direction.

celogeek commented 12 months ago

I have push on master a compatibility mode for apple books:

go install github.com/celogeek/go-comic-converter/v2@77d6600

You can try with:

go-comic-converter -applebookcompatibility

This should fix the presentation on apple books. It should be good enough on the kindle too.

Some settings are forced in this mode:

And the title on the kindle will start in the middle, and on the left for apple books.

I have other fixes to do, but let me know if it's better or if you see some other issue after the update.

manueldidonna commented 12 months ago

I have a problem that may be related to yours.

Screenshot 2023-09-23 alle 12 47 38

When I open the converted EPUB in Apple Books I get an extra page (Page 2 in the screenshot) that doesn't exist in the file. I used -titlepage=0 -hascover=0 options.

If I enable hascover I solve that problem but I get a different one: Screenshot 2023-09-23 alle 12 55 21 The cover is excluded from the pages and Apple Books treats the first page as the cover so it completely alters the layout by messing up double pages

fixed 😍 I also set -hascover=0 -titlepage=0 Thank you for the speed with which you solved the problem, you were very kind

manueldidonna commented 12 months ago

I have another suggestion to improve compatibility with Apple Books.

This is what a manga converted with this program looks like Screenshot 2023-09-24 alle 15 00 59 This is how it should be. Perhaps a solution would be to add an option to append a blank page after the title Screenshot 2023-09-24 alle 15 05 38

celogeek commented 12 months ago

To preserve the blank images included in the original content, you can use the option

-noblankimage=false

Can you try that ?

yoanhg421 commented 12 months ago

@celogeek It looks good now. Thank you.

I did noticed on the other threat, that you might look into using a package to make MOBIs. It would be great to have the performance of GoLang to also work on mobi. Right now the best solution is using kindlegen but is slow. It hope testing goes well.

Anyways. Thank you so much for your help.

Here is a screen shot of the GUI I made for the go-comic-conveter. it also calls kindlegen to convert epubs into mobi, It also looks at the CPU count to process mobi quickly, but nothing compared to Go speeds.

Screenshot 2023-09-24 at 7 46 04 AM Screenshot 2023-09-24 at 7 47 47 AM
manueldidonna commented 12 months ago

To preserve the blank images included in the original content, you can use the option

-noblankimage=false

Can you try that ?

The original content doesn't include a blank image, in the second screenshot I've added it manually

yoanhg421 commented 12 months ago

@celogeek I send a book to the Kindle App on my phone, and there are a few that don't work there.

Manga mode doesn't work. you can only go right -> , Images are not centered, with the phone on landscape image goes all the way to the left.

Since the books work great on Kindle devices, but have issues on other viewers including the Kindle App. I wonder if there is a missing setting in the book that makes it work properly and automatically. I don't read on the kindle app, so it doesn't affect me as of now. But I thought you might want to know. I was thinking on purchasing a KOBO reader, But the books might not work great there either.

Phone Portrait. book has landscape support enabled. but it's the same result in portrait only.

IMG_5035

Lanscape. IMG_5036

Amazon Manga portrait. IMG_5039

Lanscape IMG_5040

yoanhg421 commented 12 months ago

To preserve the blank images included in the original content, you can use the option

-noblankimage=false

Can you try that ?

The original content doesn't include a blank image, in the second screenshot I've added it manually

Can you add a blank jpeg to your manga if you want that layout. Some sources will include all pages including blank ones. others will only include pages that have content. It depends where you get your manga from. If celogeek adds a blank page to fix that manga, it going to break every other manga out there. That is not a software fix, that is a content fix.

It looks like you are missing a blank page.

celogeek commented 12 months ago

To preserve the blank images included in the original content, you can use the option

-noblankimage=false

Can you try that ?

The original content doesn't include a blank image, in the second screenshot I've added it manually

I guess you want the content to start on the left.

So you can may be use this combinaison

-applebookcompatibility -hascover -titlepage=0

So the cover will be out of content, no title page, and with applebookcompatibility you will start on the left instead of the right.

celogeek commented 12 months ago

@celogeek I send a book to the Kindle App on my phone, and there are a few that don't work there.

Manga mode doesn't work. you can only go right -> , Images are not centered, with the phone on landscape image goes all the way to the left.

Since the books work great on Kindle devices, but have issues on other viewers including the Kindle App. I wonder if there is a missing setting in the book that makes it work properly and automatically. I don't read on the kindle app, so it doesn't affect me as of now. But I thought you might want to know. I was thinking on purchasing a KOBO reader, But the books might not work great there either.

Phone Portrait. book has landscape support enabled. but it's the same result in portrait only.

IMG_5035

Lanscape. IMG_5036

Amazon Manga portrait. IMG_5039

Lanscape IMG_5040

Forget about Kindle App on iPhone/iPad. It's the worst viewer I've ever seen. Even if it's from amazon, they are so late compare to the Kindle device.

It support only normal mode, no spreading, and landscape is awful. Better wait for an update (don't hold your breath here), or use alternative tools like:

celogeek commented 12 months ago

@celogeek It looks good now. Thank you.

I did noticed on the other threat, that you might look into using a package to make MOBIs. It would be great to have the performance of GoLang to also work on mobi. Right now the best solution is using kindlegen but is slow. It hope testing goes well.

Anyways. Thank you so much for your help.

Here is a screen shot of the GUI I made for the go-comic-conveter. it also calls kindlegen to convert epubs into mobi, It also looks at the CPU count to process mobi quickly, but nothing compared to Go speeds.

Screenshot 2023-09-24 at 7 46 04 AM Screenshot 2023-09-24 at 7 47 47 AM

Wow looks nice. Let me know if you need a pipe mode to communicate with the tools. I can go with a JSON output for the progress bar instead, and stdout too.

yoanhg421 commented 12 months ago

That would be great. I was reading the stderr but it was making everything to slowdown because I had to regex the output every time to get the progress, but removed it and just have a spinner out.

Having a better way to communicate with your binary would be awesome. specially JSON and stdout. I need to read the progress (percent) as integer 20 || 90 and the time elapsed and remaining.

I already get the used settings and set the -quiet flag.

I'll implement the progress bar in the frontend, otherwise I have to extract the information.

I'm building your binary from source, I'm not sure if you have plans to release the cross platform binaries in the release page, but right now it have to pull the changes, build, and then update my app.

The plan is to use Github Actions so that I can also release CrossPlatform builds.

Let me know if you want to give the app a try. Right now it only works on M1 Macs.

manueldidonna commented 12 months ago

To preserve the blank images included in the original content, you can use the option

-noblankimage=false

Can you try that ?

The original content doesn't include a blank image, in the second screenshot I've added it manually

I guess you want the content to start on the left.

So you can may be use this combinaison

-applebookcompatibility -hascover -titlepage=0

So the cover will be out of content, no title page, and with applebookcompatibility you will start on the left instead of the right.

You're right, thanks

celogeek commented 12 months ago

That would be great. I was reading the stderr but it was making everything to slowdown because I had to regex the output every time to get the progress, but removed it and just have a spinner out.

Having a better way to communicate with your binary would be awesome. specially JSON and stdout. I need to read the progress (percent) as integer 20 || 90 and the time elapsed and remaining.

I already get the used settings and set the -quiet flag.

I'll implement the progress bar in the frontend, otherwise I have to extract the information.

I'm building your binary from source, I'm not sure if you have plans to release the cross platform binaries in the release page, but right now it have to pull the changes, build, and then update my app.

The plan is to use Github Actions so that I can also release CrossPlatform builds.

Let me know if you want to give the app a try. Right now it only works on M1 Macs.

I've create another issue for this one: https://github.com/celogeek/go-comic-converter/issues/22

If everything is ok on this issue, may be we can close it ?