Manga Translator

Why ?

Boy read manga.
Boy reach latest chapter.
Chapter not translated.
Boy get mad.
Boy do code instead of waiting like a normal person.

How ?

Yolo for bubble recognition and text segmentation
Open CV for bubble and masking
PIL for text replacement
Api's or neural networks for translation
Deepfillv2 for inpainting and bubble cleanup

Progress

[x] Bubble inpainting using deepfillv2
[x] Bubble recognition (should improve with more training data)
[x] Free text recognition (should improve with more training data)
[x] Bubble text extraction
[x] Bubble masking
[x] Bubble text insertion
[x] Bubble Text ocr
[x] Bubble Text translation
[x] Bubble Text hypenation
[x] Format and structure dataset
[x] Create converters i.e. yolo => coco, my dataset => yolo etc
[ ] Detection of text color
[ ] Better free text ocr
[ ] Better free text translation
[ ] Improve text resize algorithm, some texts are too small/big
[x] Implement lama for cleaning

Models

Available in the latest release

Install

setup Anaconda
clone this repository
Open a terimal in the cloned repository
Run the following

conda create -n manga-translator python=3.10
conda activate manga-translator
conda install -c conda-forge poetry

poetry install
# For cuda support run "poe force-cuda"

Download models to models/modelname (i.e. models/detection.pt)

Usage

UI

poe run-server

CLI

python main.py -f image1.png image2.png

Datasets

Detection

Segmentation

assets/examples

Original	Translated

Japanese => English Jujutsu Kaisen

Japanese => "Meow" Solo Leveling

Japanese => Clean The Rising of the Shield Hero

Japanese => English A Certain Scientific Accelerator

Japanese => English One Punch Man

Japanese => English Oshi No Ko