SethRobinson / UGT

Universal Game Translator - Uses Google's Cloud Vision to read and speak dialog from any image/game in any language
https://www.codedojo.com/?p=2426
Other
122 stars 24 forks source link

Is it possible to get the captured text into the clipboard without doing the translation? And getting the translated text without displaying the screenshot with the text on it? #39

Closed Cloudstrifeff7 closed 3 years ago

Cloudstrifeff7 commented 3 years ago

Good morning! Is it possible to get the captured text into the clipboard without doing the translation? And getting the translated text without displaying the screenshot with the text on it?

For example, I know very well that after the translation a screenshot with the translated text is displayed and you can also put it in the clipboard. But I would like an option to disable the screenshot without having to press space every time. I ask for this feature because I find it more comfortable to see the translation directly in a separate window through a program that displays the contents of the clipboard in real time, so the screenshot does not sit on top of the game window.

I also ask if I can use UGT to get the original text without doing the translation. So I get the original text in the clipboard and always display it in a separate window with another program. In the config.txt file I can't disable the translation, so I ask if it's possible. Thanks a lot in advance!

SethRobinson commented 3 years ago

Ok, I've added features that should help you use UGT this way in V0.72!

Here's how:

At this point, it should be working and putting stuff on your clipboard, without seeing or needing to close the pop-up window. (Use the cliprect, then the normal capture after that's set, same as usual)

To disable translations completely, edit config.txt and change the default language to "00". In the latest config_template.txt you can just uncomment a line to do that.

Anyway, I'll leave this issue open a bit, would like to know if it actually works since it's very poorly tested by me... !

Cloudstrifeff7 commented 3 years ago

Thank you very much! I did some testing and it works. But the no overlay mode even though it doesn't show me anything, I still have to press the button on the gamepad twice, otherwise the text isn't picked up and put in the clipboard (It's like I have to close the screenshot even though I don't see it. ). So in no overlay mode the text should put it in the clipboard by pressing the button on the gamepad only once. Finally I would like to ask you if it is possible to set the size and position of a rectangle where to grab the text, so that the whole window is avoided. Because I play with the gamepad and it is inconvenient every time with the mouse to set a rectangle. So I set the size and position of a rectangle with the mouse and then every time I press the gamepad button it only takes the text from the rectangle. This is great for those games that have the text always in the same position. Thanks a lot in advance! Finally in the config file I would also put Italian in the target language codes. I enter it manually, but I think you can put it as a language in the list.

SethRobinson commented 3 years ago

Hmm - weird, it does seem to work for me on both counts -

First, about the rect setting - if I set the rect with Alt-F10, any subsequent "capture" button pushes on my Xbox pad will only capture from the last selected rect, until it's changed. (Default is pushing in on the right stick I think) This doesn't work for you? It forgets the previously set rect? (Note: If UGT is quit and restarted, it does forget the rect)

On needing to press the button twice - on mine, if I press capture (Right stick as button by default) it captures, and as long as I wait two seconds or so, it does internally close itself and a subsequent push will start another capture, no extra click needed, just one per capture.

However, if I push BEFORE it's finished, it cancels the active capture (with no sound or feedback) - and the NEXT button press will start a new capture.

Is the problem that you are clicking before it's done and thus canceling the capture? If so, maybe I need to change things to me more intuitive.

If a capture button is hit DURING a capture it should:

  1. Play a buzz sound and refuse to cancel if it's in mid-capture/OCR, so the scan eventually will update the clipboard
  2. Cancel and restart in that single button click <-- Best idea maybe, although if you do this the clipboard won't be updated until it successfully finishes, so the interrupted capture is just lost
  3. Multi-task the captures so they can overlap and update at the same time... hmm, not possible with the way it works currently
  4. Cancel the current capture and wait, not showing anything or playing any sound <--- this is what it does now, bad

PS: I really appreciate the feedback!

Cloudstrifeff7 commented 3 years ago

Thank you so much for the response! I can set the rectangle with the mouse and then use the gamepad to translate only the text from that rectangle (as you wrote me about this feature). First of all I want to tell you that I'm using Windows 11, so I don't know if there are any compatibility issues. However, it seems that pressing the gamepad key twice was necessary because with the keyboard I was advancing through the dialogues or using ALT + TAB to switch from the game window to the notepad to see if the text was copied to the clipboard, then I have to press the gamepad key twice. However with ALT + TAB the UGT window ends up at the bottom and to bring it back to the center I have to close the program and start it.

I want to ask you if it is also possible to add the ability to drag the UGT window anywhere, because sometimes when I use ALT + TAB to switch to other windows when I return to UGT the window is at the bottom and I can no longer click on the various functions, so I have to close the program and start it again.

Plus I don't know if I can ask here, but I'll show you this example of text captured in the clipboard from the Genshin Impact game.

image

I get this text:

It brings me back to the days when I and Stanley had just started adventuring together. We'd always drink ourselves under the table the night beforesetting out.

Is it possible to get before setting? I mean the space between those two words.

Another example:

image

I get this text:

As long​ as she keeps taking her medicine, she will eventually get well.

I wanna get: As long​ as she keeps taking her medicine, she will eventually get well.

With Castlevania: Advance Collection for Steam I have a strange problem. Because when I want to set the translation rectangle with Control + F10 the function doesn't activate, as well as when I press Control + F12 to translate the whole active window. I can however translate the whole window by clicking the button from the gamepad. So even if I have the game in the window after pressing alt + enter, I can't use the keyboard to translate, only the gamepad. So I don't know why I have this problem with this game.

Thank you so much for your work on this program! I am Italian and it helps me a lot to play video games with english text!

SethRobinson commented 3 years ago

Thanks, that helps a lot - I do see the problem with it forgetting the clip-rect now as well.

Please try 0.73, I've tried to fix the main problems:

Not sure why hotkeys don't work from Castlevania yet. Hopefully with the alt-tabbing fixes you can still get a rect set and remembered at least though.

Cloudstrifeff7 commented 3 years ago

When the Castlevania Advance Collection window is active, the keyboard hotkeys do not work. So to get around the problem I translate the whole window using the button on the gamepad. Instead for Genshin Impact if I have the gamepad connected if I try to use the rect set I get select area with transparent screen, but the mouse cursor doesn't move from the center, so I disconnect the gamepad, I set the rect set with Control + F10 and then I reconnect the gamepad to use it with Genshin.

I have found a system that works with both games though. I open a folder and with this new window active I can use the hotkeys for Castlevania, then I use Control + F10 and after setting the rectangle size I go back to the Castlevania window and translate it using the gamepad button. Also for Genshin Impact this system works, so I avoid to disconnect the gamepad.

I found that when I use Alt hotkey it disables everything (but I don't have this problem with other keys). Here I set _hotkey_to_scan_draggablearea|Alt,F10| for the rectangle and I get this:

image

Thank you very much for 0.73!

SethRobinson commented 3 years ago

Hrm - on the Alt-F10 hotkey not working issue I think I know what it is - probably something else is already using it. I got the same error with Alt-F10, I figured out it is a hotkey that is set by NVidia's overlay feature (save last 5 minutes of recording) - after changing the hotkey in NVidia's hotkey settings, Alt-F10 worked in UGT.

I've put out a new version (V0.74) that gives a nice message if a hotkey can't be set and also fixes a few bugs. Thanks!

Cloudstrifeff7 commented 3 years ago

Thank you very much! The problem it was NVidia's overlay feature!