Add translation and TTS support

Lekhak123 commented 2 years ago

Hello, I tried your project and it's wonderful. Thanks for making this. I had a few ideas in my mind. I modified your code and added some simple code to :

translate the text generated from ocr model into english text
Used pyttsx3 to convert the text into speech.

So now whenever someone uses ALT+Q to use the ocr feature, this piece of code also reads the text (english translated) for us. Basically a manga reader. (pip install pyttsx3 googletrans) needs to be done.

The changed file is: Cloe/app/utils/image_io.py

_Changes were made in the imageio.py file only.

(the new code starts from here) :

"""
Poricom Image Processing Utility

Copyright (C) `2021-2022` `<Alarcon Ace Belen>`

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
"""
#pip install pyttsx3 googletrans
from io import BytesIO
from PyQt5.QtCore import QBuffer
from PyQt5.QtGui import QGuiApplication
from PIL import Image
from googletrans import Translator
import pyttsx3
translator = Translator()
engine = pyttsx3.init()
voices = engine.getProperty('voices')
rate = engine.getProperty('rate')
engine.setProperty('voice', voices[1].id)
engine.setProperty("rate", 178)
def pixboxToText(pixmap, model=None):

    buffer = QBuffer()
    buffer.open(QBuffer.ReadWrite)
    pixmap.save(buffer, "PNG")
    bytes = BytesIO(buffer.data())

    if bytes.getbuffer().nbytes == 0:
        return

    pillowImage = Image.open(bytes)
    text = ""

    if model is not None:
        text = model(pillowImage)
        translated= translator.translate(text.strip()).text
        engine.say(translated)
        engine.runAndWait()
        engine.stop()
    return text.strip()

def logText(text, mode=False, path="."):
    clipboard = QGuiApplication.clipboard()
    clipboard.setText(text)

    if mode:
        with open(path, 'a', encoding="utf-8") as fh:
            fh.write(text + "\n")

blueaxis commented 2 years ago

Hello thanks for making this pull request!

While I think this is a nice-to-have feature, there should be an option to disable translations or TTS in the settings menu. It would also be better if users can pick a better translation service such as DeepL.

I'll merge the code once you apply the proposed changes above and make this a proper PR.

blueaxis commented 2 years ago

Closing this now since there has not been any updates

blueaxis / Cloe

Add translation and TTS support #1