Linux unresponsive white UI

[axi92]

After an hour or so the UI is white and I don't know if it still doing something in the background or not.

Are there any debug logs that I can enable?

OS: Ubuntu 22.04 Package: AppImage Hash: 3f496f82fb3dcb41ae80ba8ac88b2a36d860722c

[DevilXD]

Hello,

Is this what you see right after opening the application, or it opens fine, but the UI breaks like that only after a while?

As far as debugging this goes, you'd need to run the application from source, to see if anything would be printed out to the console. Even then, ot may happen so that the text box will consume anything printed out to it, without actually displaying anything, so there's no sure way to debug anything for now.

[axi92]

It opens fine and starts collecting drops as it should and after some time it looks like this.

I am using Ubuntu as stated in the first post, is there a setup script or a documented way as I can run it from source so I can give you debug logs?

[DevilXD]

There's some instructions here: https://github.com/DevilXD/TwitchDropsMiner/wiki/Setting-up-the-environment,-building-and-running

As I said though, there's little chance any of them will actually be useful. Freezes generally leave no logs/traces.

@guihkx Do you have an idea what may be causing this?

[guihkx]

No idea, unfortunately. The symptoms of this one are similar to these other two issues, too:

• 302

• 513

The first issue looked like it was solved, but it was not. And the second one seems to be "resolved by itself".

One thing these issues have in common, is that they're happening on Debian-based distributions, but that doesn't help much either. lol

I even tried setting up Linux Mint on a virtual machine, but I could not reproduce the problem.

So it seems that the only working alternative, for now, is to run the Windows version through Wine.

[DevilXD]

Shucks. I'm starting to think that it's the makeshift busy-loop not being very liked on Linux for some reason, but testing that will be quite a challenge, if needed. Let's try something simple for now.

@axi92 Can you check if installing that emoji font mentioned in #513 helps?

[axi92]

I already did this in 2023 https://github.com/DevilXD/TwitchDropsMiner/issues/302#issuecomment-1719186159 or was it a different font?

[DevilXD]

Hmm. I don't know what to do. Would you like me to prepare a version with all emojis replaced by ASCII characters?

[axi92]

I can try that version for sure, do you think that the font is the problem?

[DevilXD]

Please try with https://github.com/DevilXD/TwitchDropsMiner/tree/no_emoji

All emojis have been replaced with ASCII characters. Please note that this was only done in the default translation, so you'll need to switch to the default English language.

Please give it a try and report back if it continues to freeze or not.

[axi92]

Is there an Appimage from that branch? I was not able to find a build from that branch.

[DevilXD]

The automatic development builds are done only for the master branch, and this is a separate, testing one. You'll need to either build it yourself, or just run the miner from source. There's some instructions on how to do that here: https://github.com/DevilXD/TwitchDropsMiner/wiki/Setting-up-the-environment,-building-and-running

[axi92]

OK, I am running it from source now. I hope it behaves the same way as the Appimage.

Edit: Still running after 2 days

[Minty-stack]

I'm also testing the no emoji version on Mint 22, and it's still running without a hitch - probably the first time I didn't have to restart it for more than a day. Manual reloading works too - fingers crossed this solves the problem completely!

[DevilXD]

Per https://bugs.python.org/issue42225, apparently Tk/Tcl may crash or freeze when displaying emoji, if the Tk version is older than 8.6.11. I'm not entirely sure how this can be checked in the auto-builds, but I saw a tk8.6-blt2.5 package being mentioned in the PyInstaller build for Linux. Not sure what happens with AppImage, and where the Tk interpreter comes from for it.

There's also this open issue: https://bugs.python.org/issue44217

This page mentions full unicode support being available in version 8.7 of Tk/Tcl: https://core.tcl-lang.org/tips/doc/trunk/tip/542.md

Newest version of Python still seems to mention bundling Tk/Tcl 8.6, so I'm not sure how to fix this. A higher version of TkInter would need to be installed as a separate package on Linux, and then Python would need to be instructed somehow to use that instead.

@guihkx Have you experimented with that by chance?

[guihkx]

https://bugs.python.org/issue42225

Interesting... It'd be great if the people who are experiencing this issue could run the following test program (taken from here), and see if it crashes/hangs:

from tkinter import Tk
from tkinter.scrolledtext import ScrolledText
root = Tk()
text = ScrolledText(root, width=80, height=40)
text.pack()

for i in range(0x10000, 0x40000, 32):
    chars = ''.join(chr(i+j) for j in range(32))
    text.insert('insert', f"{hex(i)} {chars}\n")

input("Press enter to exit")

It doesn't crash nor hangs here with Python 3.12.5 and Tk/Tcl 8.6.14, and the emojis show up correctly with colors (most of them begin to show up in the 0x1f300 row).

I'm not entirely sure how this can be checked in the auto-builds, but I saw a tk8.6-blt2.5 package being mentioned in the PyInstaller build for Linux. Not sure what happens with AppImage, and where the Tk interpreter comes from for it.

The latest PyInstaller build on Linux includes Python 3.10.14 and Tk/Tcl 8.6.10.

The latest AppImage build includes Python 3.10.12 and Tk/Tcl 8.6.12, which in theory shouldn't have the issue you linked.

This page mentions full unicode support being available in version 8.7 of Tk/Tcl Newest version of Python still seems to mention bundling Tk/Tcl 8.6, so I'm not sure how to fix this.

As far as I understand, there hasn't been a stable release of Tk/Tcl newer than 8.6 yet, so it might be tricky to get a pre-built 8.7 release. But even if we do, I'm not sure it has backward compatibility with 8.6...

[DevilXD]

Huh. Apparently early Tk/Tcl 8.7 versions were released back in 2019, so I'd imagined they've pushed out a stable version by now, but apparently not. Version 8.6.14 appears to be the latest release.

[Minty-stack]

I just ran that program guihkx posted, and it did display, didn't hang or anything, and I was able to scroll down to the very bottom and back up again. Let me leave it open over night and see if it hangs then - the miner would normally be responsive and working for me for anywhere between 4 to 10 hours before it froze, so let's see if this can replicate it.

EDIT: The program stayed on through the night, but it stayed responsive - could still scroll up and down now, and everything was showing correctly. Hmm...

[axi92]

After 2,5 days its unresponsive. I started it from source but nothing in the terminal.

It did not respond to SIGTERM I had to send a SIGKILL.

[Minty-stack]

Mine just hung too, after around the same time - 2 days or so. It was trying to unsuccessfully mine the sub purchase drop for Warhammer 40k Space Marine 2, with a lot of channels available, similarly to the screen above with World of Tanks. Just theorizing, but can the cause of instability be associated with too many channels available?

[guihkx]

Just theorizing, but can the cause of instability be associated with too many channels available?

I'm not sure about that. I've left the miner running for much longer than 2 days but never had that happen to me.

Out of curiosity, do both of you have automatic suspend to RAM (or disk) enabled when the system goes idle?

[Minty-stack]

I actually have no idea how to check that on Mint (apologies, still very much a noob). Looking at power management settings, the "suspend when inactive for" option is set no "never" - would that be it, or is there another setting somewhere (or, more likely, a terminal command for it)?

[guihkx]

I think that setting is managed by systemd "behind the scenes" on Linux Mint, so you could probably check the status of the relevant systemd units with:

systemctl status {sleep,suspend,hibernate,hybrid-sleep}.target

[Minty-stack]

Here's what I get by running that command:

○ sleep.target - Sleep
     Loaded: loaded (/usr/lib/systemd/system/sleep.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

○ suspend.target - Suspend
     Loaded: loaded (/usr/lib/systemd/system/suspend.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

○ hibernate.target - System Hibernation
     Loaded: loaded (/usr/lib/systemd/system/hibernate.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

○ hybrid-sleep.target - Hybrid Suspend+Hibernate
     Loaded: loaded (/usr/lib/systemd/system/hybrid-sleep.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

It does seem like everything is inactive.

[guihkx]

Yeah, that was a shot in the dark pretty much...

If I could reproduce the issue, I'd try to attach gdb (a debugger) to the PID of the running, frozen process, and then interrupt it (with either a SIGTERM or SIGKILL). After that, I'd try to figure out where/why it froze by looking at the stack trace with the bt command.

Unfortunately, that's a pretty advanced topic which even I'm not an expert in, lol.

Not to mention it's not even guaranteed we'd be getting useful information from it, especially if we don't have the appropriate debug symbols installed, which I'm not sure it's as easy to do on Ubuntu when compared to Arch, where we'd just use debuginfod.

But perhaps you guys can run the app in debug mode with logging enabled (i.e. python main.py -vvvv --log), and then wait for the app to freeze again. After that, you could inspect the very end of the log.txt file (created in the same folder of main.py), and see if there's anything useful there; or if the last lines match for both of you...

[!CAUTION] Just make sure to never share the entire log.txt file here, because it contains HIGHLY sensitive information that can be used to steal your Twitch account.

[DevilXD]

I'm both glad and disappointed that it's not the emojis causing this. If they were, I'd need to either remove them completely, or introduce a way to disable those somehow. On the other hand, it's seems they're not the culprit, so we're back to square one with the issue =/

I've actually had a talk regarding the debug mode the other day, and figured that it could be improved, to automatically expunge the authentication token out of the logged messages. That'd make it not require such a warning to be provided in the first place.

[Minty-stack]

I used to run the miner with -vvv reflexively every time, and in my case it would always hand after a reload operation (similar to the manual reload crash I reported earlier and that magically fixed itself). This was the case yesterday too, if I remember correctly.

Since I missed multiple drops I wanted to get, I just gave up for now and I'm running the Windows version through Wine now - seems to be working without a hitch, let's see if anything happens in a couple of days.

[axi92]

I got an error but I don't think it is related to this issue? Should I create a new one for this? Run from source 802f2506b34892d8db0523850f58b46f44a62c1e

2024-09-06 14:01:52.835:      DEBUG:    Request: (method='GET', url=URL('https://video-weaver.cdg10.hls.ttvnw.net/v1/playlist/....m3u8'), kwargs={'headers': {'Connection': 'close'}})
2024-09-06 14:01:55.911:       INFO:    synop_s goes OFFLINE
2024-09-06 14:01:57.003:       CALL:    Channel update from websocket: synop_s
2024-09-06 14:01:59.593:       CALL:    Channel update from websocket: a_big_koko
2024-09-06 14:02:26.169:       CALL:    Drop update from websocket: Epic Community Chest (Albion Online, 98/180)
2024-09-06 14:02:52.319:      DEBUG:    Response: 404: <ClientResponse(https://video-weaver.cdg10.hls.ttvnw.net/v1/playlist/.....m3u8) [404 Not Found]>
<CIMultiDictProxy('Content-Type': 'text/plain; charset=utf-8', 'Vary': 'Accept-Encoding', 'x-content-type-options': 'nosniff', 'x-error-message': 'failed to fetch weaver playlist: channel is offline', 'Date': 'Fri, 06 Sep 2024 12:02:52 GMT', 'Content-Length': '10', 'Access-Control-Allow-Origin': '*', 'Connection': 'close')>

2024-09-06 14:02:52.320:      DEBUG:    Request: (method='HEAD', url='Not Found', kwargs={})
2024-09-06 14:02:52.320:      ERROR:    Exception in _watch_loop task
Traceback (most recent call last):
  File "/home/user/Workspace/git/TwitchDropsMiner/utils.py", line 132, in wrapper
    await afunc(*args, **kwargs)
  File "/home/user/Workspace/git/TwitchDropsMiner/twitch.py", line 860, in _watch_loop
    succeeded: bool = await channel.send_watch()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Workspace/git/TwitchDropsMiner/channel.py", line 385, in send_watch
    async with self._twitch.request("HEAD", stream_chunk_url) as head_response:
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Workspace/git/TwitchDropsMiner/twitch.py", line 1285, in request
    response = await self.gui.coro_unless_closed(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Workspace/git/TwitchDropsMiner/gui.py", line 2147, in coro_unless_closed
    return await next(iter(done))
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Workspace/git/TwitchDropsMiner/env/lib/python3.12/site-packages/aiohttp/client.py", line 1337, in send
    return self._coro.send(arg)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Workspace/git/TwitchDropsMiner/env/lib/python3.12/site-packages/aiohttp/client.py", line 586, in _request
    raise err_exc_cls(url)
aiohttp.client_exceptions.InvalidUrlClientError: Not%20Found
2024-09-06 14:02:52.326:       INFO:    Websocket[0] stopped.
2024-09-06 14:02:52.327:       INFO:    Websocket[1] stopped.
2024-09-06 14:02:52.327:       INFO:    Websocket[2] stopped.

[guihkx]

aiohttp.client_exceptions.InvalidUrlClientError: Not%20Found

That looks more like #504

And it probably happened due to very unlucky timing

[DevilXD]

Yeah, that looks like #504. You haven't happened to grab the output printed right above this, have you?

[axi92]

Sure I have the full log in the log file. How much do you need? I will paste it in #504

Anything to redact there?

[DevilXD]

I'll respond under #504.

[DevilXD]

A lot of changes has happened during the past 2 weeks, including fixes to some parts of the miner not working as they should.

Is this issue still present? What's the current status of this?

[Minty-stack]

I actually switched to Manjaro a week and a bit ago, and the miner has been working flawlessly - no freezes at all. I do git pull when there are updates and restart it, but it works way better than it did on Mint, looks different too - all the emojis show correctly, and in color. I'm in the middle of something right now, but I'll try booting back to Mint tomorrow and running the latest version for a while to see if it plays nice now.

[Minty-stack]

Happy to report that the latest version of the miner has been running on Mint for more than a day with no issues - it always hung way sooner before, so the problem might have been solved by some of the recent changes!

[DevilXD]

Huh. That's honestly puzzling for me. The only fixes/changes that I'd imagine have an impact on this would be:

• Fix asyncio.as_completed not cancelling its tasks (only matters in case of fatal exceptions happening in a specific place(s) in the code)
• Implement GQL rate limiter (slows down the rate of GQL requests)
• Fix can_watch ignoring drops_enabled status (could've ended up switching to a channel where drops wouldn't be enabled, preventing it from gaining mining progress)

None of these in particular point at a possible issue that could happen only on a specific Linux distro. The rest of the commits have to do either with syntax corrections, typing changes, documentation changes, or code cleanup, things that aren't really impactful on the working side of the miner, but instead more about general code maintenance.

If you can, please run it for a little bit longer, I'd like to see if this issue really is truly gone now.

[Minty-stack]

Sure, will do! It's still running smooth by the way - tomorrow morning if it stays up it will be the first time ever it goes for over 48 hours without freezing - fingers crossed!

[axi92]

@Minty-stack are you running it from appimage or from source?

[Minty-stack]

@Minty-stack are you running it from appimage or from source?

From source - did a git pull right before launching it at 10AM yesterday and it's still going strong as we speak - just did a manual reload to confirm, and yup, going strong.

Edit: still looks different than Manjaro though - the present emoji and such is black and white...

[Minty-stack]

Last report before I boot back to Majaro, since I left something there I want to get back to - miner is still working fine, no freezes or crashes - seems the problem has been solved! At least on my end that is - it has never lasted this long without restarting before.

[DevilXD]

Well... Alright then.

I'm closing this as solved, even though I have no idea what the problem was. If the issue would resurface, please open another issue.