search

DevilXD / TwitchDropsMiner

An app that allows you to AFK mine timed Twitch drops, with automatic drop claiming and channel switching.
MIT License
1.74k stars 170 forks source link

Fatal error crash: [{'message': 'service error', 'path': ['game']}] #415

Open Unclepotap4ik opened 10 months ago

Unclepotap4ik commented 10 months ago

The program crashes in few seconds after opening with text:

07:32:37: Fatal error encountered: 07:32:37: 07:32:37: Traceback (most recent call last): 07:32:37: File "main.py", line 158, in main 07:32:37: File "twitch.py", line 764, in run 07:32:37: File "twitch.py", line 916, in _run 07:32:37: File "twitch.py", line 1709, in get_live_streams 07:32:37: File "twitch.py", line 1566, in gql_request 07:32:37: exceptions.MinerException: GQL error: [{'message': 'service error', 'path': ['game']}]

Try to reinstall, but that doesn't work

DevilXD commented 10 months ago

Hello. "service error" signifies a problem with Twitch internals, not the miner. These usually go away on their own after some time passes, once Twitch fixes the issue.

Which games you have added to your priority list?

Asiimov2077 commented 10 months ago

This error also occurs when I farm RUST drop.

DevilXD commented 10 months ago

@daeeros Auto-restart makes no sense for a project like this, as it's going to run into the very same error right away, and all you end up with, is the miner spamming the Twitch internal API with errors - a very sure way of encouraging them to take more and further steps to bot-proof their system and essentially kill this project. If auto-restart would be a good idea, I'd implement it ages ago.

Please don't do this.

someC0d3r commented 10 months ago

Can confirm:

image

09:56:48: Fatal error encountered:
09:56:48: 
09:56:48: Traceback (most recent call last):
09:56:48:   File "C:\Users\Administrator\Desktop\TwitchDropsMiner\TwitchDropsMiner-master (1)\TwitchDropsMiner-master\main.py", line 158, in main
09:56:48:     await client.run()
09:56:48:   File "C:\Users\Administrator\Desktop\TwitchDropsMiner\TwitchDropsMiner-master (1)\TwitchDropsMiner-master\twitch.py", line 764, in run
09:56:48:     await self._run()
09:56:48:   File "C:\Users\Administrator\Desktop\TwitchDropsMiner\TwitchDropsMiner-master (1)\TwitchDropsMiner-master\twitch.py", line 916, in _run
09:56:48:     new_channels.update(await self.get_live_streams(game))
09:56:48:                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
09:56:48:   File "C:\Users\Administrator\Desktop\TwitchDropsMiner\TwitchDropsMiner-master (1)\TwitchDropsMiner-master\twitch.py", line 1709, in get_live_streams
09:56:48:     response = await self.gql_request(
09:56:48:                ^^^^^^^^^^^^^^^^^^^^^^^
09:56:48:   File "C:\Users\Administrator\Desktop\TwitchDropsMiner\TwitchDropsMiner-master (1)\TwitchDropsMiner-master\twitch.py", line 1566, in gql_request
09:56:48:     raise MinerException(f"GQL error: {response_json['errors']}")
09:56:48: exceptions.MinerException: GQL error: [{'message': 'service error', 'path': ['game']}]
09:56:48: 
09:56:48: Exiting...
09:56:48: 
09:56:48: Application Terminated.
09:56:48: Close the window to exit the application.

On latest master version

Infos:

image

image

pip updated requirements updated

Vadson159 commented 10 months ago

@DevilXD I guess topics like this will occur from time to time anyway. Because of the very same errors I was unlucky to skip a few Halo Infinite drop campaigns that were running in my nighttime and I couldn't restart miner manually. Maybe it would be a good idea to output this error for the user as "twitch having troubles right now, waiting..." or something like that, and make it auto restart in a minute or two to not spam twitch API that hard.

DevilXD commented 10 months ago

When a "service error" occurs, there's nothing you can do to help - restarting won't fix it. This is an internal Twitch error that cannot be "handled" by any other way, than just waiting (even a couple of hours) and then restarting the miner. Handling this in a more graceful way was tracked by #347 for some time, but then it got closed due to the issue being no longer present (until again now).

Regardless, I've noticed that unrequested application crashes could use some more notification noise to them, which https://github.com/DevilXD/TwitchDropsMiner/commit/caecef4ecd6f97b103355f7a929a094c71fbc794 should help with.

I'll try to debug this for Rust specifically in the mean time. So far I haven't been able to reproduce it, Rust starts mining just fine for me.

someC0d3r commented 10 months ago

@DevilXD

Can you tell what this method call does? Does it log to a file by any chance? If so I maybe can provide you my response JSON :/

image

wlk3773208 commented 10 months ago

In the past, this error would only occur occasionally and the frequency was very low. But this time, the frequency of the error is very frequent. This error will definitely occur here within 2 hours. Could it be caused by someone abusing the mechanism? For example, divide 100 systems on a powerful server and then use it for 100 accounts at the same time?

DevilXD commented 10 months ago

Can you tell what this method call does? Does it log to a file by any chance? If so I maybe can provide you my response JSON :/

@someC0d3r That's a debug logger call. You'd need to run the miner with the GQL debug flag: --debug-gql. Also use --log to log everything to a file.

The logging system was never fleshed out enough for me to consider using it over "live" debugging via the in-built VSCode debugger, but I guess it's worth a shot.

A word of caution though: note that the resulting log may include some sensitive information about your Twitch account, that'd allow a 3rd party to authorize as you, which could result in "unpleasant" consequences (general "your account has been hacked or hijacked" / potential loss of account). I never built into it any system that'd filter those out. The log file should thus either be thoroughly examined by you, to remove any traces of sensitive information (you have to have the knowledge of what to look for), and any omission means risking the safety of your Twitch account, which means that some "more personal" way of giving me the log file would be strongly preferred. I personally have zero to no interest accessing someone else's account, but if you don't trust me with the log file either, I'd suggest simply not sending one at all.

If you have Discord, I'm available under @devilxd. Otherwise, you can email me the log file to 4180725+DevilXD@users.noreply.github.com.

DevilXD commented 10 months ago

@wlk3773208 I've seen others setup true account farms, that'd run this miner (among other ones) on thousands of accounts. This is considered not only Twitch API abuse, but also an abuse of this project, which I do not support in the slightest. For a more descriptive journey about this, you can check out the project goals: https://github.com/DevilXD/TwitchDropsMiner#project-goals All of this contributes to Twitch putting out preventive measures, that makes it harder to run a personal, single-account miner like this one. But there isn't much I can do to combat this, unfortunately.

Regarding the issue, it's most likely an extra condition on Twitch side, which I'm yet to discover. Since I can't reproduce the error myself, there isn't much I can do for now. I'll keep trying, I guess.

boypipe commented 10 months ago

@wlk3773208 I've seen others setup true account farms, that'd run this miner (among other ones) on thousands of accounts. This is considered not only Twitch API abuse, but also an abuse of this project, which I do not support in the slightest. For a more descriptive journey about this, you can check out the project goals: https://github.com/DevilXD/TwitchDropsMiner#project-goals All of this contributes to Twitch putting out preventive measures, that makes it harder to run a personal, single-account miner like this one. But there isn't much I can do to combat this, unfortunately.

Regarding the issue, it's most likely an extra condition on Twitch side, which I'm yet to discover. Since I can't reproduce the error myself, there isn't much I can do for now. I'll keep trying, I guess.

It happens when running on VM and using the manual built solution (adding a line of code) provided by another person in issue #386

DevilXD commented 10 months ago

Please tell me it's not verify=False, because that's a pretty terrible idea / """solution""". A better idea would be to pip install either certifi or truststore to supply the missing certificates instead.

Unclepotap4ik commented 10 months ago

Looks like problem is gone. I have only one crash this morning, and then program works very well

DevilXD commented 10 months ago

I bet it was just Twitch themselves messing something up internally on their side, and this being the byproduct. I should really get to some proper handling for "service error" specifically. This isn't one where retrying after mere seconds makes any sense, the miner will need to essentially enter idle and wait until it's over. I mentioned this already under #347, but it got closed without adding said handling, so that was pretty bad on my part.

Zhanyee commented 1 month ago

I'm having this problem for around 2 months now, is there a solution?

JourneyOver commented 1 month ago

There def seems to be an issue with Destiny 2 today. In the past two hours, three separate tickets have been closed as duplicates of the same problem, and I'm encountering the same error—I'm sure others are too.

For now, the only solution is to either add Destiny 2 to the exclusion list or remove it from the priority list if you're using "priority list only" mode. Doing either seems to resolve the issue for the time being.

Taizunz commented 1 month ago

Just started happening again today

00:41:08: Fatal error encountered:
00:41:08: 
00:41:08: Traceback (most recent call last):
00:41:08:   File "main.py", line 155, in main
00:41:08:   File "twitch.py", line 587, in run
00:41:08:   File "twitch.py", line 747, in _run
00:41:08:   File "twitch.py", line 1656, in bulk_check_online
00:41:08:   File "asyncio\tasks.py", line 571, in _wait_for_one
00:41:08:   File "twitch.py", line 1387, in gql_request
00:41:08: exceptions.GQLException: [{'message': 'service error', 'path': ['channel', 'viewerDropCampaigns']}]
00:41:08: 
00:41:08: Exiting...
dogodogowoof commented 1 month ago

Can confirm adding Destiny 2 to my exclusion list worked. thx @JourneyOver

DevilXD commented 1 month ago

I've put more effort into investigating this issue, by checking how the Twitch website handles this. I found out that the DropCampaignDetails GQL call on the website has a different hash than the miner was using, so I've updated it: https://github.com/DevilXD/TwitchDropsMiner/commit/f11e3347e04d2fbf1f6811d8edf45733ca90c7e7

Please let me know if that resolves the issue for now.

JourneyOver commented 1 month ago

So far it looks to be good! :D

Luckz commented 1 month ago

I've put more effort into investigating this issue, by checking how the Twitch website handles this. I found out that the DropCampaignDetails GQL call on the website has a different hash than the miner was using, so I've updated it: f11e334

Please let me know if that resolves the issue for now.

On e79ab9d, if I don't block Destiny 2, I still get:

18:55:30: Traceback (most recent call last):
18:55:30:   File "<path>\main.py", line 155, in main
18:55:30:     await client.run()
18:55:30:   File "<path>\twitch.py", line 587, in run
18:55:30:     await self._run()
18:55:30:   File "<path>\twitch.py", line 747, in _run
18:55:30:     await self.bulk_check_online(acl_channels)
18:55:30:   File "<path>\twitch.py", line 1656, in bulk_check_online
18:55:30:     response_list = await coro
18:55:30:                     ^^^^^^^^^^
18:55:30:   File "<path>\Lib\asyncio\tasks.py", line 631, in _wait_for_one
18:55:30:     return f.result()  # May raise f.exception().
18:55:30:            ^^^^^^^^^^
18:55:30:   File "<path>\twitch.py", line 1387, in gql_request
18:55:30:     raise GQLException(response_json['errors'])
18:55:30: exceptions.GQLException: [{'message': 'service error', 'path': ['channel', 'viewerDropCampaigns']}]
JourneyOver commented 1 month ago

hmm wonder why you are still able to run into the issue, as I tested the latest commit on an account with just destiny 2 in the priority list as well as testing with all three different priority modes and haven't ran back into the issue anymore since these last 2 commits so I'm wondering if you have something else possibly going on..

DevilXD commented 1 month ago

Twitch seems to have some kind of a rolling deployment going on. Some people experience a crash, some not. Some get the updated hash value, some are still left running the old one. Eventually though, the change is migrated to all users. The timeframe for this appears to be at least a few days, I'd say 2-3.

The error seems to be originating from the bulk status check now. I'll investigate further.

EDIT: The error pointed at the "AvailableDrops" section of the bulk status check, so I've checked the GQL hash, and it was outdated as well. Please try again with the latest master.

JourneyOver commented 1 month ago

Feels like we really need to find a way to update the hashes automatically some how periodically or something just so that we stop running into issues due to a hash being out of date xD Makes me wonder if there are any other hashes that are out of date that we don't know about yet.

dogodogowoof commented 1 month ago

@DevilXD now working for me with https://github.com/DevilXD/TwitchDropsMiner/commit/f128e918d2ac666a19b8f03e3de66b9a55c1aacd AND its mining the destiny 2 drop.

DevilXD commented 1 month ago

@dogodogowoof Cheers for verifying =)

@JourneyOver Outdated hashes don't do anything, until Twitch starts to change around their internals. Once they do so, they usually change the query around as well, and then hash changes. The result of this is either the service crashing when using the old hash (service error), or the query stops being cached on the server side (PersistedQueryNotFound).

I don't really know where the hashes come from, but I know that getting that information is nigh impossible. I know it's the raw GQL query being "compressed" into a hash form, where both the client and the server know the actual query, and by sending the hash itself, both can relay the information of what the client wants, and what the server is supposed to respond with, with minimal information being actually sent as the request. Technically, I could send raw GQL queries without hashes, but that'd only get rid of the "PersistedQueryNotFound" error, and it'd still crash when Twitch would change around their internal structures.

The only solution would be to do the impossible and analyze the web client to extract hashes out of it, but the problem is that it's minified JS with some quite cryptic pieces of code, or it's straight up encrypted/mangled JS that specifically targets being hard to reverse-engineer. The simplest solution for now is to just update the hashes manually, as they don't really change that often. For example, the last time the AvailableDrops hash was updated, beyond yesterday, was 17 months ago: https://github.com/DevilXD/TwitchDropsMiner/commit/5b8b6c7fdda0061a813e968f6ccd23117d42fa84

Other examples beyond the updates from the last 2 days:

All of the above operations, plus the "Inventory" hash, were updated 20 months ago before that: https://github.com/DevilXD/TwitchDropsMiner/commit/bfd389392127488a963fc696583fc140745494b8

If you'd like to investigate on your own, here are the two main files I'm pretty sure are responsible for the Twitch web client functionality:

JourneyOver commented 1 month ago

Hmm I might take a look at things when I have some free time to really dig down into it, but yea I guess if they don't change as often as I thought they was going to then it might just be pointless.

skapytek commented 1 month ago

For me this error occurs every time im starting twitchdropsminer

Luckz commented 1 month ago

For me this error occurs every time im starting twitchdropsminer

Temporarily add Lost Ark and Black Desert to your exclusion list?

skapytek commented 1 month ago

For me this error occurs every time im starting twitchdropsminer

Temporarily add Lost Ark and Black Desert to your exclusion list?

still the same error