celogeek / go-comic-converter

Convert CBZ/CBR/Dir into Epub for e-reader devices (Kindle Devices, ...)
MIT License
61 stars 8 forks source link

Display the picture in its original size #38

Open ssbroad opened 1 month ago

ssbroad commented 1 month ago

The resolution of many comics is smaller than the resolution of KS - 1860x2480

If you enlarge the picture to adapt to the height of the screen, it will cause a certain amount of blur.

I want to display the image in a point-to-point (pixel for pixel) manner, i.e. display the image smaller than the screen resolution in its original size on the screen.

For example, the resolution of this image is 1445x2048.

0010

I tried changing the image size in xhtml but it had no effect.

<img src="../Images/img_1_p0.png" alt="Image 1 Part 0" style="width:1749px; height:2479px; top:0.02%; left:0"/>

=>

<img src="../Images/img_1_p0.png" alt="Image 1 Part 0" style="width:1445px; height:2048px; top:0.02%; left:0"/>

celogeek commented 1 month ago

kindle size is limited to the size of the SR profile. When you put more like the KS profile, going from ePub to MOBI (using the website or the local tools), will reencode the images to smaller resolution.

Usually the fastest / best render is the SR profile.

To support bigger resolution:

ssbroad commented 1 month ago

I found a strange bug.

I want to keep the original size of the image without any cropping.

Depending on the output format, we will get 2 results.

check it: original image 2000-2000-Claymore v1-036


no-crop & jpg f-jpg


no-crop & png f-png

Why does choosing jpg format output result in cropping part of the content?

I have disabled the cropping parameter

celogeek commented 1 month ago

can you share a zip with the command line you use ?

ssbroad commented 1 month ago

Claymore v1.zip

jpg -input "Claymore v1.zip" -output "Claymore v1.epub" -profile KS -manga -titlepage 0 -noblankimage=0 -sort 2 -strip -crop=0

png -input "Claymore v1.zip" -output "Claymore v1.epub" -profile KS -manga -titlepage 0 -noblankimage=0 -sort 2 -strip -crop=0 -format "png"

ssbroad commented 1 month ago

There is no problem with epub, the file that makes the difference is in the final mobi

I tried to use ChainLP to reproduce this bug, but the mobi converted using ChainLP (jpg qualty=85) did not have this problem.

ChainLP and go-comic use the same kindlegen.exe trans manga

ssbroad commented 1 month ago

Claymore v1 from ChainLP https://pixeldrain.com/u/9G7hXapJ Claymore v1 from Go-Comic https://pixeldrain.com/u/B5AYzbAr The epub output by ChainLP does not have this problem. After epub2mobi, both jpg and png can retain complete pictures

This Bug is really strange. If you have any ideas and need my help, please don't hesitate.

celogeek commented 1 month ago

The only thing I can imagine is the PNG produce by GCC has an option epub2mobi doesn't like and have trouble to work with.

ssbroad commented 1 month ago

Isn't it the jpg that has the problem? Some edges of jpg were incorrectly cropped during conversion to mobi.

ssbroad commented 1 month ago

There is no problem converting jpg to mobi.

Observed in landscape mode, the jpg retains the complete original image.

The problem is that in portrait mode, the image seems to be forcibly enlarged by the epub css, resulting in the inability to display the complete content.

Of course, the discarded content is blank pixels, but disabling cropping means we need to display the full content.

ssbroad commented 1 month ago

Does your KS also have the problem that jpg cannot handle disabled cropping correctly?

https://github.com/user-attachments/files/15821018/Claymore.v1.zip

It's really weird. It's obvious that the picture saves the complete content, but KS always seems to accurately discard the blank part at the edge.

celogeek commented 1 month ago

Can you set the aspect ratio to 1.6. I think kindle doesn’t handle well other aspect. That may be the case here. I will check soon. Just need to find some time to work on it. I see the aspect ratio is missing in your command.

ssbroad commented 1 month ago

Test aspect-ratio-1.6 and bug still exit. The really weird thing about this error is that the image files in both epub and mobi retain their original size. It shows that the '-crop=0' has been executed well. But our ks device did crop the image on its own. It's like the kindle system has an automatic cropping function, even though we know it's impossible

celogeek commented 1 month ago

I think it just misplace the image. I have a bit of time, I'm checking...

celogeek commented 1 month ago

When I compare the generation of the file (epub) with your options between png and jpg, both are strictly the same, except the format of the output image.

Sending to my kindle to see the result.

celogeek commented 1 month ago

Ok I see. png source looks like the image is not resize. And jpg it enlarge on the screen.

The jpg is the one intend by my sizing option. It's support to spread as much as possible.

so something from the epub to mobi have issue in the convertion.

this format was intend to return as much details as possible on the image (lost less), but not made for kindle. It's more for readers that support ePub with png natively.

I will try to see if I can extract the content of the mobi somewhere.

ssbroad commented 1 month ago

I have used calibre to extract the image in mobi. There is no problem with the image file and it is exactly the same as the image in ePub.

celogeek commented 1 month ago

I extract the content of the mobi, I see some diff like:

-    <img src="../Images/image00006.jpeg" alt="Image 1 Part 0" style="width:1653px; height:2480px; top:0.00%; left:5.56%" width="1333" height="2000"/>
+    <img src="../Images/image00006.gif" alt="Image 1 Part 0" style="width:1653px; height:2480px; top:0.00%; left:5.56%" width="1333" height="2000"/>

The png is output to GIF instead of JPG. I suspect the kindle to have a more limited support to this format. GIF is strange to convert to.

Also, here the format:

➜  file jpg-mobi/mobi8/OEBPS/Images/*
jpg-mobi/mobi8/OEBPS/Images/cover00007.jpeg: JPEG image data, JFIF standard 1.01, resolution (DPI), density 1x1, segment length 16, baseline, precision 8, 1279x1920, components 3
jpg-mobi/mobi8/OEBPS/Images/image00006.jpeg: JPEG image data, JFIF standard 1.01, resolution (DPI), density 1x1, segment length 16, baseline, precision 8, 1279x1920, components 1

➜  file png-mobi/mobi8/OEBPS/Images/*
png-mobi/mobi8/OEBPS/Images/cover00007.gif: GIF image data, version 89a, 1279 x 1920
png-mobi/mobi8/OEBPS/Images/image00006.gif: GIF image data, version 89a, 1279 x 1920

As you can see, mobi convertion reduce the image anyway. that's why the SR profile is the best, because the resizing is done locally, and then AWS keep the original image without resizing it again. It's way faster for a better result (you can choose the quality !)

celogeek commented 1 month ago

Will retry the same test + extract, with SR profile.

celogeek commented 1 month ago

similar diff:

-    <img src="../Images/image00006.jpeg" alt="Image 1 Part 0" style="width:1200px; height:1800px; top:3.12%; left:0.00%" width="1200" height="1800"/>
+    <img src="../Images/image00006.gif" alt="Image 1 Part 0" style="width:1200px; height:1800px; top:3.12%; left:0.00%" width="1200" height="1800"/>

But the resolution change a bit. sending to my kindle ....

celogeek commented 1 month ago

same result. png is converted to gif on kindle. and it doesn't display it as it should with the jpg.

I think I should revamp the profile with capability.

For example for kindle device:

if you select desktop, then I can relax the settings to allow more option (as it depend of the reader). and on kobo it's another list of compatibility

I have already something for Apple Books, with enforce settings. It should be a profile.

ssbroad commented 1 month ago

https://pixeldrain.com/u/STXC64LL - Claymore from ChainLP.epub https://pixeldrain.com/u/cCDMHDL6 - Claymore from ChainLP.mobi Can you check these 2 files using KS? I'm very curious why epub made using ChainLP don't have this problem. The jpg image is complete on the kindle without any cropping.

ssbroad commented 4 weeks ago

After my tests I can conclude that the problem is with the image files generated by Go-Comic.

I swapped the image files in Go-Comic.epub and ChainLP.epub

Here is chainLP img set ChainLP Img Set

Result: Go-Comic.epub (jpg from ChainLP) ChainLP.epub (jpg from Go-Comic) Go-Comic.epub is normal but there is a problem with ChainLP.epub

ImgCropBug.zip

I really hope you can fix this bug after carefully comparing the differences between these 2 jpg files

Here's a quick fix. Add a ‘-raw‘ parameter.

If the user disables cropping they obviously want to keep the full image. In this case, we can directly copy the original image without performing any processing on the image. Unless destructive changes are made to the image, such as split, rotation, resize, contrast, brightness.

celogeek commented 4 weeks ago

The 2 images here:

from ChainLP.jpg:   JPEG image data, JFIF standard 1.01, resolution (DPI), density 96x96, segment length 16, baseline, precision 8, 1333x2000, components 1
from Go-Comic.jpeg: JPEG image data, baseline, precision 8, 1333x2000, components 1

the ChainLP generate a JFIF image. I've seen that trick for writing AZW file already. It seems mobi is more or less compatible with JPEG. he kind of need a fixed header.

I will try to use that standard. It's not by default. We trying to generate an image that probably have more than 20 years ... mobi is old.

celogeek commented 4 weeks ago

I create this branch: https://github.com/celogeek/go-comic-converter/tree/38-display-the-picture-in-its-original-size

Can you try to see if I manage to fix the issue ?

go install github.com/celogeek/go-comic-converter/v2@d1265a51350446f646840a2f20f9652fa89b8a8d
ssbroad commented 4 weeks ago

The problem still exists. After replacing it with chainLP.jpg, the problem disappears.

It seems we haven't found the real difference between these 2 jpg's yet.

jpg from new branch but crop bug still exists. from new branch

celogeek commented 4 weeks ago

I though it was ok, I've test it. I may have take the wrong file. will retry.

is it the cover above ?

ssbroad commented 4 weeks ago

Yes, the image above is from the latest branch. By the way, a recent discovery: If cropping is disabled, only the cover image does show the full size in kindle.

ssbroad commented 4 weeks ago

This gave me a guess that the problem lies in the cropping detection program part. Even if the user disables cropping, the program still leaves traces in the image. Only the cover image skips detection. But this cannot explain the return to normal in landscape mode. Well, that's just my guess.

celogeek commented 3 weeks ago

The cover has his own way to display on kindle. It doesn't use the CSS style. The image use CSS that will scale up the image to match the kindle size. The aspect ratio will discard (by the kindle) blank part of the image in Portrait mode. The landscape has another rendering view, it have some space between each side and a different aspect as the portrait.

I'm checking if any crop has been done even if we ask not to do so. I'm checking if cropping is actually not a cropping but more a kindle behavior. You say you replace image from another source. How did you recompile the ePub ? you just zip it again ? ePub has a special header, so a simple zip is not a valid one. I don't know if that matter for kindle converter.

Also, on your screen shot, it's hard to tell what do you mean by "crop". I don't see it obviously on the screen shot.

celogeek commented 3 weeks ago

I'm rechecking the files:

On GCC, I see on that, the aspect ratio = 1.5. I think it's set to automatic and pick up the most common size observed in the files. On CLP, I don't see the aspect ratio settings. It's not set (original-size = X*Y). I guess it's fall back to a default value that will render differently.

Will try other stuff, with image swapping to see it on device.

celogeek commented 3 weeks ago

OMG I think I've found the issue !

The aspect ratio is support to readjust it properly, but it doesn't save the computing. So the result is that we keep the size of the device itself. And aspect is 1.33 instead of 1.6. So it may explain the wrong settings.

Fixing and retrying...

celogeek commented 3 weeks ago

I've fix aspect ratio computing:

go install github.com/celogeek/go-comic-converter/v2@fee8a037dc16faf9e58d3f64191b5080593e883e

So the device has 1.33 aspect ratio: 1860x2480 And amazon advice to use 1.6 aspect ratio, meaning the image should fit inside this box: 1550x2480

Here:

    <meta name="viewport" content="width=1550,height=2480"/>
...
    <img src="../Images/img_1_p0.jpeg" alt="Image 1 Part 0" style="width:1550px; height:2326px; top:3.10%; left:0.00%"/>

So the image is scale to fit into the box, and we move it to center it.

It maximise the rendering: Image

celogeek commented 3 weeks ago

The only way to keep original size and position inside the device box, is I think to create an option for that. This way I will use the original size for image, and use Top/Left to place it in the box.

I see I've got a "resize" option. which if turn on, will resize the image if it exceed the device size. If it turn off, it will keep the original resolution.

In both case it will adapt the CSS to maximise the rendering.

I will play a bit with the img style, see what happen if I don't change the image size.

celogeek commented 3 weeks ago

I try to keep the original size, and even put the Top: X% correctly to center. But it's like the kindle completley ignore my CSS. I suspect the "position: absolute" only works on landscape and is ignored in portrait.

The strange part is that the png which is convert to gif seems to apply properly the css.

celogeek commented 3 weeks ago

I will try an alternative jpg encoder to see if that's change anything. I feel the render on portrait is great, but also totally ignore any custom css I want to use. So it's almost like showing the img directly.

The gif respect the size vs viewport, but the kindle ignore the CSS for absolute position. It lead to a small image on the left instead of in the center.

So next I will try an alternative jpeg encoder. Then I may review my css, it seems pretty useless in that state.

celogeek commented 3 weeks ago

I've test an alternative jpeg encoder mozJpeg (or libjpeg-turbo). I've got the same result unfortunately.

On portrait, the css is ignore and the image is extended respecting the aspect ratio to the maximum size. On landscape, the css seems to be respected. But may be not all of them.

When I run kindlegen, I can see this:

Warning(htmlprocessor):W28003: Value specified for CSS property in content is not supported by Kindle readers. Please refer Kindle Publishing Guidelines about usage of property: 'position: absolute' in file: /var/folders/bn/vrxfysxs67lb5gznk1hk2k6m0000gp/T/mobi-fBLXMZ/OEBPS/Text/style.css

So I guess it's really not supported.

Here if you want to give a try to libjpeg-turbo:

go install github.com/celogeek/go-comic-converter/v2@f33c095094c4066420715e279dc84edb8f3be166

This create smaller and more optimized jpeg. It's also faster. But it doesn't support Palette, so the cover need to be encoded with the generic library.

I may add a flag to activate this if people prefer this encoder. But it required an external lib.

celogeek commented 3 weeks ago

Image 2

It clearly works on landscape. I try putting original size here. try removing viewport.

ssbroad commented 3 weeks ago

I just unzip the epub as a zip and replace the images in it. Although it may not be a formal epub, it does work! ! You can try.

Bug-Image

replace-zip(epub)-image-to-ChainLP's

By the way, the image suffix generated by ChainLP is jpg, while go-comic is jpeg.

All Your Need Test.zip

I also thought at first that the problem was caused by the wrong proportion setting. The wrong proportions caused the Kindle to discard the white space around the image.

But this doesn't explain the picture below:

2000-new-boundaries

/////////

Why-kindle-can-recognize-new-boundaries

I just added some elements to the bottom of the original image, and the kindle immediately recognized the new boundaries accurately. And discarded the little white space at the bottom.

I think it's the image and not the css that's causing the error:

  1. After replacing the image generated by ChainLP (other parts in epub have not been changed, such as css, content.opf), the problem is solved.
  2. Kindle can recognize new boundaries. Kindle obviously does not have a cropping function. This can only be explained by the fact that go-comic's cropping program has some unknown effect, although the user has disabled cropping.
celogeek commented 2 weeks ago

When GCC read the source, it will decode it in memory. Then it go through a bunch of filter:

Then the result is encoded to jpg or png using the official jpeg or png module include in the go standard library. I tried also the facebook library jpeg-turbo (mozjpeg), which optimize the file to take less spaces. Both have the same result, the CSS is ignore for some reason.

I think the kindle detect the jpg and do a special treatment in portrait when we use the comic layout. But when the jpg is not recognise (or same if it's a gif format) then it skip the special treatment and fallback to the CSS defined.

In any case, the position absolute I use seems not supported, so I will try to have a similar rendering without this option.

I need to play a bit with the CSS. I already try it in the past, but the result where not as nice as today.

celogeek commented 2 weeks ago

As an alternative, I can create an image of the size of the device and place the jpg exactly where I need. This way I'm sure the css will be respected. I can even remove it.

ssbroad commented 2 weeks ago

You are right, the problem may indeed be with kindlegen.

Your instructions helped me find a perfect solution.

Bug-Fix

The problem is solved!

Guess what I did?

Well, I just added some long bars with a width of 1 pixel each in the lower left corner and the upper right corner of the original image.

1 pixel 1 pixel A 1 pixel B

I highly recommend this approach instead of creating device-sized images. Adding pixel occlusion barely changes the comic content, while the latter breaks image alignment in landscape mode.

A more perfect approach would be to perform whitespace detection on the image and then add 1 pixel occlusion only in the whitespace areas.

ssbroad commented 2 weeks ago

Simply put, it is to detect whether the pixels in the bottom left corner and upper right corner of the image are white. If so, change the pixel to black to prevent it from being cropped to achieve a true no-cropping mode. Maybe 1 pixel is enough.