j3soon / whisper-to-input

An Android keyboard that performs speech-to-text (STT/ASR) with OpenAI Whisper and input the recognized text; Supports English, Chinese, Japanese, etc. and even mixed languages.
27 stars 1 forks source link

Better exception handling and logging #17

Open j3soon opened 7 months ago

j3soon commented 7 months ago

As mentioned in the offline reported by @ijsun, the current keyboard will fail silently after using it for a while.

When encountered this issue, a simple Clear All to close all apps can reliably fix the issue by resetting the keyboard. Unfortunately, the steps to reproduce this issue is remained unknown. (not sure if it's due to unreliable internet connection)

This kind of nondeterministic silent failure is difficult to debug. Therefore, we should somehow add a logging mechanism that can report all kinds of silent exceptions and maybe save it in a second page of the keyboard settings app, allowing the user to go through it for debugging purposes. (It's better to allow debugging directly on user's phone without the need of using adb or other external PC tools)

Potentially related:

2023/12/02 Update: Clearing the cache in the app settings page can also fix the issue by resetting the keyboard.

j3soon commented 7 months ago

Without thinking too much, I guess this issue may be related to the Exception thrown here:

https://github.com/j3soon/whisper-to-input/blob/6495487afec37a4d47511c8a0917395cb8ecd308/android/app/src/main/java/com/example/whispertoinput/WhisperTranscriber.kt#L76-L78

The exception may cause the onStartTranscription to fail and make the WhisperInputService requiring a reset to work again:

https://github.com/j3soon/whisper-to-input/blob/6495487afec37a4d47511c8a0917395cb8ecd308/android/app/src/main/java/com/example/whispertoinput/WhisperInputService.kt#L114-L123

A potential solution is to not throw an exception, but display a Toast message instead. This is just my random guess though...

Another observation by @ijsun is that the amplitude feature still works when the issue occurs.