search

xulihang / ImageTrans-docs

Documentation of ImageTrans, a computer-aided image translation tool. ImageTrans的文档项目。ImageTrans是一款计算机辅助图片/漫画翻译软件。
https://imagetrans.readthedocs.io/
85 stars 9 forks source link

ScreenReader improvement #521

Closed Kamikadashi closed 9 months ago

Kamikadashi commented 10 months ago

It would be great if the ScreenReader had the capability to adjust the opacity of the ‘fixed area capture’ window. Additionally, it would also be beneficial if it could monitor changes in the selected area and automatically perform OCR on the area if the changes exceed a specified threshold. This enhancement would broaden the potential use cases for the ScreenReader, making it a more versatile tool.

lhj5426 commented 10 months ago

嘻嘻 希望可以联动下这个 https://github.com/xulihang/ImageTrans-docs/issues/237 都是在一个界面上的功能 要是能一起迭代 优化就好了

xulihang commented 10 months ago

Monitoring the changes is doable using things like motion detection which I have experimented: https://www.dynamsoft.com/codepool/motion-detection-for-barcode-scanning.html.

Could you also name the game for which you desire to use the screen reader so that I can have a try?

Kamikadashi commented 10 months ago

I've done some additional testing and it seems that for the mangaTranslator OCR to accurately detect text in old, low-resolution games, it needs to be upscaled first. I suppose that’s why mangaTranslator now has a 4x-UltraSharp upscaler. I’m not certain how good it is, but I’ve done some testing with realesrgan-ncnn-vulkan and it consistently helps in achieving good results. There’s a possibility that the newer mit48px model might work fine without upscaling, but I haven’t done enough testing yet to be sure.

With new high-resolution games, the OCR works just fine right out of the box, with no need for upscaling. The only issue I’ve encountered with high-res so far is that when there’s a full screen of text with many lines stacked one below the other, they sometimes get scrambled. It would be beneficial to have an option to force them to be ordered by the x or y-axis.

For low-res, I've tested with: 魔人形 (Neko Project II emulator) and マテリアル・ガール High-res: Geminism 〜げみにずむ〜

xulihang commented 9 months ago

These features are added in v2.8.0.

  1. You can adjust the alpha of the window in fixed area mode.
  2. You can start an interval to OCR an area. Motion detection is enabled by default. You can adjust them in the preferences.
xulihang commented 9 months ago

image

Kamikadashi commented 9 months ago

For some reason, it takes a screenshot with the box on top of the selected area and doesn’t sort the lines correctly, even though I checked the ‘Correct Text Order’ option. chrome_jkC0mo4z5c

xulihang commented 9 months ago

Which OCR are you using? The order should be correct using manga-image-translator.

If it is not the desired order, try to set the sorting criteria to vertical ordinate in the project settings. You can test the sorting in the main window first.

xulihang commented 9 months ago

Hiding the capture box in the interval mode is annoying, so I just keep it. You can set its alpha value to 0 so that the captured image will not have it.