search

zyddnys / manga-image-translator

Translate manga/image 一键翻译各类图片内文字 https://cotrans.touhou.ai/
https://cotrans.touhou.ai/
GNU General Public License v3.0
5.39k stars 557 forks source link

New Inpainting #58

Open mspykerez opened 2 years ago

mspykerez commented 2 years ago

You think this is a good Inpainting alternative to have https://github.com/msxie92/MangaInpainting ?

Based on the sample images it looks to do a tremendous job...

mspykerez commented 2 years ago

Also would like to ask the status of improving the english translation text which currently is displayed in a way that is not visually appealing... probably due to the all vertical/horizontal text issue.

For the lack of a better solution would it be possible to just display the english text on a neat way like the extension ''Raw Manga Fan'' does https://chrome.google.com/webstore/detail/raw-manga-fan/lmojhocblnlnjboabeengdkgogipihmg and just proceed to automatically and cleverly take a screenshot of the results akin of the ''GoFullPage - Full Page Screen Capture'' extension https://chrome.google.com/webstore/detail/gofullpage-full-page-scre/fdpohaocaechififmbbbbbknoalclacl which appears to be able to reconstruct the image to its full size even if it happens to be shown out of screen...

Snowad14 commented 2 years ago

I am surprised by the quality of how the text is shaped by the extension. But I can tell you that the "lama" impainting (which zyddnys has implemented but which is not yet used) is better than the one you gave by my personal experience.

mspykerez commented 2 years ago

Yes it's really nice, the best I've seen really. The automatic placement of the text, font selection style, size and color and how they managed to keep the background damage to a minimum and even reconstruct it on some cases to near original state it's truly amazing. Unfortunately they do not have a desktop app and the extension only works if you are online, it uses DeepL for the translation I believe. Thou you can load and batch translate local images you don't appear to be able to save them with the translations on top which is what I'm looking to do.

Btw the Inpainting tool I linked on the first post is not related to the extension, it's just something I found and thought if it would be worth checking out.

Snowad14 commented 2 years ago

I'm looking for exactly the same thing. But this extension is on the server side, so there is no guarantee that it will remain free and functional

mspykerez commented 2 years ago

It's been that way for a long time now, my only surprise is that not many people know about it.

There are other extensions like 'Ichigo Reader' which has some premium features https://chrome.google.com/webstore/detail/ichigo-reader%F0%9F%8D%93-translate/lepcfgkehgeiblekejomdmdklmjdmflp but atm it kinda butchers the image by just erasing the japanese text by simply putting a white background over it, although I hear the author is looking into improving that part. The good news is that it has an option to save the images with the translations on top.

The funny part is that the most sucky extension of them all is the most popular and that is 'Manga Translator' https://chrome.google.com/webstore/detail/manga-translator/oooikipggipaohjibkmoiacmmlilgokn must be because of the catchy name.

Anyway you think it is because the technical way zyddnys is doing the all detection-replacement of the japanese text in order to make the images savable with the translation that visually it is just not as good as the ''Raw Manga Fan'' extension which in turn is also because of that that you can't save the images with this extension? That's why I thought if the 'screenshot' method would be an approach to consider taking instead. I mean 'Raw Manga Fan' seems to use some sort of overlay on top of the image and not draw anything into it, you can see it by mouseover the translation and it will instantly switch to the raw text and vice-versa on that part of the image.

Snowad14 commented 2 years ago

It's not really a problem like that I think, it's just that the project is made to be used on vertical texts (which he is very good at)

mspykerez commented 2 years ago

If that is the case does that mean it will need an huge overhaul? Like new data and training and all that?

Snowad14 commented 2 years ago

No, this project is very good with OCR, impainting, but is really not good with text rendering (python file you can see). It needs a lot of work to rework it

mspykerez commented 2 years ago

Not sure if they are using the same data, RMF has been around for a year. That being said can you make the english translation output look as nice as RMF :)

mspykerez commented 2 years ago

You can aslo save the output of that extension (save as .png) and calculate mse with inpaint result of ballontranslator.

I'm confused... are you saying that I can save the images that RMF produces with the english translation on top? How?

dmMaze commented 2 years ago

Oh, sorry, I mistakenly upload all images in that directory to it and took them as RMF's output ;) I'll see what I can do with the eng typesetting in the following month