Makememo / MemoAI

MemoAI Video to translated text, subtitles and notes made easy.
https://memo.ac
502 stars 6 forks source link

字幕會顯示錯誤的時間 #257

Closed WingCH closed 5 months ago

WingCH commented 6 months ago

Screenshots image

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

sdugoten commented 6 months ago

same issue using v1.3.0

sdugoten commented 6 months ago

You can use this to reproduce

https://www.youtube.com/watch?v=CmFIcAn0PeQ

image

image

The problem is that when the original line is long, the translation might drop to the next slot...which make the whole translation slip

sdugoten commented 5 months ago

version 1.3.2 still having this problem using the same youtube link above to test.

joyshan1986 commented 5 months ago

version 1.3.2 still having this problem for video over 40 minutes

Makememo commented 5 months ago

https://github.com/Makememo/MemoAI/releases/tag/v1.3.3

sdugoten commented 5 months ago

https://github.com/Makememo/MemoAI/releases/tag/v1.3.3

  • When transcribing, select cuda mode for GPU mode
  • Go to Settings-Lab-Turn on the flash attention mechanism switch

  • 轉寫時,GPU 模式選擇 cuda 模式
  • 前往設置 - 實驗室 - 打開 flash attention 機制開關

I just tested the provided sample https://www.youtube.com/watch?v=CmFIcAn0PeQ and the translation is off from the begining

20240528_220143

20240528_220129 20240528_220112

The problem seems like missing translation on the red square above and everything slip

sdugoten commented 5 months ago

20240528_222004

And then I tried to use Large(v3) to test again and translate, something wierd happen. You see the translation all group into 1 line...

I think you can use that youtube link to try a couple times on translation, you will see there are lot of missing translation, or the translation just slip to next line

sdugoten commented 5 months ago

20240528_223136

I understand you group a couple lines together for translation to save cost, which make sense. I think you might try to do something in your coding like the picture above and see if you can mark the number for each line a bit more accurate

sdugoten commented 5 months ago

BTW, please reopen this case, I am pretty sure this is not fixed yet after a couple tries

@Makememo

Makememo commented 5 months ago

BTW, please reopen this case, I am pretty sure this is not fixed yet after a couple tries

@Makememo

His problem is a transliteration problem. The problem of your translation misalignment is an AI problem, and frankly there is no 100% solution, we tried your way, JSON output, and various methods. Did not have the desired effect. So we provide single sentences and continued translation to alleviate this dislocation.