search

Amiron49 / Aris.Moe.Ocr.Translation.Overlay

Overlays translation over a chosen screen area using OCR
MIT License
8 stars 4 forks source link

Completely replace the raw text with the translated text and have it saved to the HD #43

Open mspykerez opened 2 years ago

mspykerez commented 2 years ago

I tested the Chrome Extension and found it a bit confusing that the translated text is put on top of the original RAW text without 'erasing' it first. I wish you did something like Raw Manga Fan instead which tries to completely remove the RAW text from the picture and replaces it with the translated text using a nice font and all while at the same time providing a nice translation quality.

I'd also ask if it would be possible to somehow load images in batches (multiple images at the same time from my local computer), have them translated and after that save them back to the disk with the translation over them?

Amiron49 commented 2 years ago

Thanks for reaching out.

Background replacement: I was already thinking about how to better render the text over the image but the way they do it is pretty smart. They also render "text boxes" on top of the original but at the same time provide a calculated background image per box to mask the original text. I will have to research a bit on how to achieve that first but implementing that will probably take a lot of time. Except if there are ready made solutions that I can just use. But yeah I'd have to look around first.

Better text rendering: That is something that should be quicker to do. The current implementation is the most lazy and quickest one I was able to do and improving the size, font and placement is a smaller task that could be done in a few weeks/days.

These two I'll gladly work on.

Batch saving of private images:

This one is a bit more complicated. I'm currently paying Deepl and Google for every image that gets translated by the extension. As long as these images are public I'm happy to cough up a few hundred per month, as long as it "serves" the general public. Which in turn means that I don't want to pay for private translation efforts out of my own pocket. I'd consider giving the downloadable GUI a feature for batch translating private images but that would mean that you'd have to provide your own API keys for Deepl and Google (Or fall back to Tesseract), which means shouldering the cost yourself. Would that solve your use case ?

mspykerez commented 2 years ago

I only mentioned Raw Manga Fan because as far I can tell it gives the best results (OCR & translation quality) but there are other extensions that also do the same job like Anity and Ichigo Reader. I do not know if they all work the same way, just figured I'd mention it so you can take a look if you want. Anity has nice OCR but the translation is really bad, as for Ichigo Reader unfortunately it only allows to translate a limited number of pages before you have to pay.

There are even more extensions like that, perhaps what is also worth mentioning in case you don't know about already is the desktop application called ImageTrans which seems to have everything I would want but is not free. They explain pretty well how it works and even show it on some videos, theres also some documentation on it here https://imagetrans.readthedocs.io/en/latest/

As for translating private images, if I use any other extension I can just go to the chrome extension options and enable it to access local files so I can just drag the image to chrome and translate it with no hassle. The problem is that you need to do this process manually 1 by 1 (not in batches) and not only that, and the biggest issue that I have with all the extensions that I encountered is if you right click the image with the intend of saving it to your computer with the translated text on it it doesn't do that and always saves the original image with the raw text instead.

I get that nowadays DeepL is the way to go but ain't there any other alternatives that you could use for free and keep DeepL as an option to the user if he wishes to use his own DeepL subscription? You could also provide support for offline translators like LEC Power Translator, it is by no means perfect, but it's better than nothing. btw there seems to be work arounds to bypass DeepL limitations while using it for free, popular tool to translate visual novels Textractor uses one of these work arounds.

One last thing, you might be interested in joining this discord https://discord.gg/XFbWSjMHJh It's from a guy that is developing multiple tools to translate manga and games using OCR. He already has some functioning tools and is always taking help from anyone who knows how to code. You could merge your project with his or you could ask to use his translation model for offline translation (he is training an AI model by feeding it human made translations from visual novels, manga, etc, and at its current state it's giving better results than google translate, goal is to surpass DeepL).