Ayanaminn / N46Whisper

Whisper based Japanese subtitle generator
MIT License
1.54k stars 127 forks source link

Add retry function for Google Gemini AI Translation #105

Closed machinewu closed 2 months ago

machinewu commented 2 months ago

修改点有三:

  1. 增加针对限流报错的retry功能
  2. 增加忽略掉安全过滤器的设置
  3. 请求报错返回None的时候,return一个">>>>> UnknownError"的字符串而不是None,防止后面字符串处理时候报错而使程序停止



把报错日志打印出来分析了一下,常遇到这4种情况:

  3%|▎         | 4/154 [00:12<07:45,  3.11s/it]未知错误,用户可以尝试查看报错信息并在Repository里提交issue
Unknown Error, please check the error log and open an issue in the repository: The `response.text` quick accessor only works when the response contains a valid `Part`, but none was returned. Check the `candidate.safety_ratings` to see if the response was blocked.
None

 14%|█▎        | 21/154 [01:00<04:34,  2.06s/it]WARNING:tornado.access:429 POST /v1beta/models/gemini-pro:generateContent?%24alt=json%3Benum-encoding%3Dint (127.0.0.1) 1364.62ms
未知错误,用户可以尝试查看报错信息并在Repository里提交issue
Unknown Error, please check the error log and open an issue in the repository: 429 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?%24alt=json%3Benum-encoding%3Dint: Resource has been exhausted (e.g. check quota).
None

 91%|█████████▏| 190/208 [08:48<00:47,  2.65s/it]未知错误,用户可以尝试查看报错信息并在Repository里提交issue
Unknown Error, please check the error log and open an issue in the repository: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
None

 84%|████████▎ | 174/208 [08:02<01:25,  2.52s/it]未知错误,用户可以尝试查看报错信息并在Repository里提交issue
Unknown Error, please check the error log and open an issue in the repository: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
None

第1种情况,是因为有些字幕触发了Gemini的安全过滤器导致的。根据Google Gemini API文档,可以设置把各类别的安全过滤器设置为“无”来解决。 第2种情况,估计是请求过快导致限流了,可以加个time.sleep解决。但是每句翻译都sleep的话会导致处理时间多很多。由于被限流的请求占比不高,所以在retry里面设置sleep就好。 第3~4种情况,一般是出现情况2后下一次retry时出现的,忽略掉再再retry就好了。所以,PR里我设置了retry 6次,实际只有3次retry有效。

Ayanaminn commented 2 months ago

感谢