machinewrapped / gpt-subtrans

Open Source project using LLMs to translate SRT subtitles
Other
310 stars 36 forks source link

GUI crashes with a segfault on Linux #134

Closed xpufx closed 3 months ago

xpufx commented 4 months ago

I can't tell when it crashes but it always crashes. Usually before finishing the current translation. It sometimes crashes while it's just sitting there which makes me think this is likely a gui library thing rather than something about the core of the app itself. I am on opensuse tumbleweed, running a session in Wayland. Will try to provide more info. (Just tested under X11. Same result.)

machinewrapped commented 4 months ago

Is this on the latest version with the default command-line arguments? I updated PySide6 in that update to the release because I was getting random GUI crashes too (on Windows). I haven't had any since then, but I'll test it some more.

Glad to hear it at least runs on Linux, wasn't sure if anybody had tried it 😅

xpufx commented 4 months ago

Oh it runs great on Linux. Thanks. :)

So I tested this on at least two latest releases. I was on the latest before you pushed the default parameters for Gemini. Unfortunately the issues happens on both. --debug doesn't really show much. Neither does strace show something that I can identify as an issue and like I said it is likely in the gui library since it is not likely that the python code itself would cause a memory fault.

xpufx commented 4 months ago

I looked up how to debug python C library segfaults. Is this of any use? (Probably not. The line referred is just QApplication's app.exec()). I want to test it in a vm. I have been having other issues with my KDE sessions.

main github/gpt-subtrans$ python -q -X faulthandler ./gui-subtrans.py INFO: Initialising log INFO: Loaded settings from /home/username/.config/GPTSubtrans/settings.json INFO: Loading stylesheet from /home/username/Code/github/gpt-subtrans/theme/subtrans.qss INFO: GUI-Subtrans v0.6.1 INFO: Logging to /home/username/.config/GPTSubtrans/gui-subtrans.log INFO: Reading project data from /home/username/Downloads/montalbano/e3-ita.subtrans INFO: Project file loaded, saving backup copy Fatal Python error: Segmentation fault

Thread 0x00007fefe9c006c0 (most recent call first): File "/home/username/Code/github/gpt-subtrans/GUI/Widgets/LogWindow.py", line 43 in AppendLogMessage File "/home/username/Code/github/gpt-subtrans/GUI/Widgets/LogWindow.py", line 65 in emit File "/usr/lib64/python3.11/logging/init.py", line 978 in handle File "/usr/lib64/python3.11/logging/init.py", line 1706 in callHandlers File "/usr/lib64/python3.11/logging/init.py", line 1644 in handle File "/usr/lib64/python3.11/logging/init.py", line 1634 in _log File "/usr/lib64/python3.11/logging/init.py", line 1489 in info File "/usr/lib64/python3.11/logging/init.py", line 2148 in info File "/home/username/Code/github/gpt-subtrans/PySubtitle/SubtitleProject.py", line 80 in InitialiseProject File "/home/username/Code/github/gpt-subtrans/GUI/FileCommands.py", line 28 in execute File "/home/username/Code/github/gpt-subtrans/GUI/Command.py", line 57 in run

Current thread 0x00007feffc15c540 (most recent call first): File "/home/username/Code/github/gpt-subtrans/./gui-subtrans.py", line 129 in INFO: Writing project data to /home/username/Downloads/montalbano/e3-ita.subtrans-backup

Extension modules: xxsubtype, shiboken6.Shiboken, PySide6.QtCore, PySide6.QtGui, PySide6.QtWidgets, regex._regex, charset_normalizer.md, google._upb._message, grpc._cython.cygrpc (total: 9) [1] 8179 segmentation fault (core dumped) python -q -X faulthandler ./gui-subtrans.py.

machinewrapped commented 4 months ago

Yeah, that's helpful - looks like a problem with the log window, which is unexpected! Maybe something to do with the character set.

It looks like QtLogger provides a handler function to call in case of errors (luckily Github Copilot knew that and auto-completed the code). I've submitted it as a speculative fix.

machinewrapped commented 4 months ago

Oh, since you're on linux I guess you wouldn't have got the PySide6 update in the last pre-release package. To make sure you're on the latest version you can activate the virtual environment with source ./envsubtrans/bin/activate and run pip install --upgrade PySide6 to make sure you've got the latest version. (should be 6.6.2).

xpufx commented 4 months ago

I think I already have it.

Requirement already satisfied: PySide6-Addons==6.6.2 in ./envsubtrans/lib64/python3.11/site-packages (from PySide6) (6.6.2)
xpufx commented 4 months ago

Oh, since you're on linux I guess you wouldn't have got the PySide6 update in the last pre-release package. To make sure you're on the latest version you can activate the virtual environment with source ./envsubtrans/bin/activate and run pip install --upgrade PySide6 to make sure you've got the latest version. (should be 6.6.2).

With the latest code the GUI stayed up much longer than before but eventually segfaulted. I don't know how to reproduce, it's erratic.

machinewrapped commented 3 months ago

Hmm, well at least it's progress. If you're able to get another crash dump it would certainly be useful in tracking down the issue.

I'm working on an overhaul of the viewmodel updates, to avoid "rebuild the view" operations as much as possible... maybe it will improve stability.

As an interim measure I'd recommend setting the scene threshold on your projects low to generate more scenes with fewer batches, so that it autosaves the project more frequently and there is less chance of losing work if it crashes. I really should add an option to save after every batch!

xpufx commented 3 months ago

New crash dump: (I did a git pull right before this)

FO: Translating scene number 1
Fatal Python error: Segmentation fault

Thread 0x00007f68a34006c0 (most recent call first):
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/grpc/_channel.py", line 1157 in _blocking
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/grpc/_channel.py", line 1173 in __call__
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/api_core/grpc_helpers.py", line 76 in error_remapped_callable
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/api_core/timeout.py", line 120 in func_with_timeout
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 144 in retry_target
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/api_core/retry/retry_unary.py", line 293 in retry_wrapped_func
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131 in __call__
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/ai/generativelanguage_v1beta/services/model_service/client.py", line 677 in list_models
  File "/home/username/Code/github/gpt-subtrans/envsubtrans/lib64/python3.11/site-packages/google/generativeai/models.py", line 190 in list_models
  File "/home/username/Code/github/gpt-subtrans/PySubtitle/Providers/Provider_Gemini.py", line 100 in <listcomp>
  File "/home/username/Code/github/gpt-subtrans/PySubtitle/Providers/Provider_Gemini.py", line 100 in _get_gemini_models
  File "/home/username/Code/github/gpt-subtrans/PySubtitle/Providers/Provider_Gemini.py", line 104 in _get_true_name
  File "/home/username/Code/github/gpt-subtrans/PySubtitle/Providers/Provider_Gemini.py", line 49 in GetTranslationClient
  File "/home/username/Code/github/gpt-subtrans/PySubtitle/SubtitleTranslator.py", line 60 in __init__
  File "/home/username/Code/github/gpt-subtrans/GUI/ProjectCommands.py", line 341 in execute
  File "/home/username/Code/github/gpt-subtrans/GUI/Command.py", line 58 in run

Current thread 0x00007f6911617540 (most recent call first):
  File "/home/username/Code/github/gpt-subtrans/./gui-subtrans.py", line 129 in <module>

Extension modules: xxsubtype, shiboken6.Shiboken, PySide6.QtCore, PySide6.QtGui, PySide6.QtWidgets, regex._regex, charset_normalizer.md, google._upb._message, grpc._cython.cygrpc (total: 9)
[1]    16583 segmentation fault (core dumped)  python -q -X faulthandler ./gui-subtrans.py
machinewrapped commented 3 months ago

Thanks - looks like a timeout in the Gemini API, something that shouldn't really crash... maybe because it happened in a list comprehension? I'll see if I can do anything to harden it.

machinewrapped commented 3 months ago

If you sync to latest you might find it improves stability - it should no longer fetch the Gemini model list for each translation request, which at least reduces opportunities for that particular API call to segfault!

machinewrapped commented 3 months ago

Hmm, stability actually seems to have gone down, not up! I suspect a deadlock on the UI thread.

machinewrapped commented 3 months ago

Submitted a highly speculative fix - not confident this is "the fix" but I've translated two full movies with Gemini without a crash since I added it.

xpufx commented 3 months ago

I have likewise translated one episode of a TV series without a crash just now. Great job. !

machinewrapped commented 3 months ago

Great :-)

I did get one crash the other day but it definitely seems more stable again. so I'll close this issue as "fixed" - please feel free to create another issue if you encounter more problems though!