search

Windows200000 / TwitchDropsMiner-updated

An app that allows you to AFK mine timed Twitch drops, with automatic drop claiming and channel switching.
MIT License
259 stars 16 forks source link

Duplicate mining #139

Open scryptio opened 3 days ago

scryptio commented 3 days ago

Description

Not sure but I would guess the miner is either not reading the inventory correctly or twitch reporting wrong things again. Since yesterday (guessed) the miner tries to mine already completed drops (i.e. overwatch, lost ark, ... in my case). It looks like it happens for all drops on me and I have to move items in the list so something which can be mined is being mined.

To Reproduce

  1. Add Overwatch / Lost Ark / xxx
  2. Let it mine
  3. Let it claim
  4. Watch it trying to start mining the same thing again

Expected behavior

Recognizes claimed / open drops properly

Observed behavior

Duplicate mining

Screenshots

No response

Logs

No response

OS

Windows 10

Build

source

Version/Commit

84f48827baf6397771ac37ef591be9a72f15c8de

Additional context

No response

MCGitHub15 commented 2 days ago

Seems like #140 is similar to your issue. Something to do with the recent drops being set years ago.

Windows200000 commented 2 days ago

@RainOne @notNSANE @gibbed @MCGitHub15 @scryptio I have a bit of time during the weekend, so I will look at it tmrw.

A GQL query log, as described in the bug request form (-vvv --debug-gql --debug-ws --log) would be hugely helpful here, so I can see what data Twitch is sending the miner, once I get to work diagnosing the issue.

barbiehunter commented 2 days ago

would be hugely helpful here, so I can see what data Twitch is sending the miner, once I get to work diagnosing the issue.

I'am not 100% Sure what i can show without Risk and what not so i "Censored" some stuff myself. Of what i understand and see, the Websocker trys many many times in 1 second to connect somewhere but failed and can't get a Update Drop amount the current Drops that you Have. 

23:02:00: CALL: No drop update from the websocket received
23:02:00: Progress: 1/30 - XDefiant, XDefiant Preseason - W6
23:02:00: CALL: Drop progress from active search: booster (XDefiant, 1/30)
23:02:10: DEBUG: GQL Request: {'operationName': 'PlaybackAccessToken', 'extensions': {'persistedQuery': {'version': 1, 'sha256Hash': 'Hash Number'}}, 'variables': {'isLive': True, 'login': 'rossburger', 'isVod': False, 'vodID': '', 'playerType': 'site'}}
23:02:10: DEBUG: GQL Response: {'data': {'streamPlaybackAccessToken': {'value': '{"adblock":false,"authorization":{"forbidden":false,"reason":""},"blackout_enabled":false,"channel":"rossburger","channel_id":153720970,"chansub":{"restricted_bitrates":[],"view_until":1924905600},"ci_gb":false,"geoblock_reason":"","device_id":"My Device ID","expires":1719609729,"extended_history_allowed":false,"game":"","hide_ads":false,"https_required":true,"mature":false,"partner":false,"platform":"web","player_type":"site","private":{"allowed_to_view":true},"privileged":false,"role":"","server_ads":true,"show_ads":true,"subscriber":false,"turbo":false,"user_id":ID,"user_ip":"My IP","version":2}', 'signature': 'signature1234', 'authorization': {'isForbidden': False, 'forbiddenReasonCode': 'NONE', '__typename': 'PlaybackAccessTokenAuthorization'}, '__typename': 'PlaybackAccessToken'}}, 'extensions': {'durationMilliseconds': 79, 'operationName': 'PlaybackAccessToken', 'requestID': 'Requested ID'}}
gibbed commented 2 days ago

@RainOne @notNSANE @gibbed @MCGitHub15 @scryptio I have a bit of time during the weekend, so I will look at it tmrw.

A GQL query log, as described in the bug request form (-vvv --debug-gql --debug-ws --log) would be hugely helpful here, so I can see what data Twitch is sending the miner, once I get to work diagnosing the issue.

One of the bugged drops (everything looks normal here):

                        {
                            "id": "cf5a9c38-2cd9-11ef-9841-0a58a9feac02",
                            "requiredSubs": 0,
                            "benefitEdges": [
                                {
                                    "benefit": {
                                        "id": "9620a89d-b07c-4160-a851-ae832fa6985e_CUSTOM_ID_1834208",
                                        "createdAt": "2024-06-17T18:46:51.501Z",
                                        "entitlementLimit": 1,
                                        "game": {
                                            "id": "515025",
                                            "name": "Overwatch 2",
                                            "__typename": "Game"
                                        },
                                        "imageAssetURL": "https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/863fe76c-33e0-45af-9e54-9b52f7f226e2.png",
                                        "isIosAvailable": false,
                                        "name": "Rose Gold Mercy Icon",
                                        "ownerOrganization": {
                                            "id": "9620a89d-b07c-4160-a851-ae832fa6985e",
                                            "name": "Blizzard",
                                            "__typename": "Organization"
                                        },
                                        "__typename": "DropBenefit"
                                    },
                                    "entitlementLimit": 1,
                                    "__typename": "DropBenefitEdge"
                                }
                            ],
                            "endAt": "2024-06-30T06:58:59.999Z",
                            "name": "1",
                            "preconditionDrops": null,
                            "requiredMinutesWatched": 180,
                            "startAt": "2024-06-25T18:00:00Z",
                            "__typename": "TimeBasedDrop"
                        },

My state for this drop:

                        {
                            "game": null,
                            "id": "9620a89d-b07c-4160-a851-ae832fa6985e_CUSTOM_ID_1834208",
                            "imageURL": "https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/863fe76c-33e0-45af-9e54-9b52f7f226e2.png",
                            "isConnected": true,
                            "lastAwardedAt": "0001-01-01T00:00:00Z",
                            "name": "Rose Gold Mercy Icon",
                            "requiredAccountLink": "https://account.battle.net/connections",
                            "totalCount": 1,
                            "__typename": "UserDropReward"
                        },

lastAwardedAt is clearly bugged on Twitch's end.

I have 48 gameEventDrops with a bugged lastAwardedAt.

uniquoo commented 2 days ago

Having the exact same issue aswell. Tried reconnecting accounts. Its on Twitchs end. I've also lost multiple drops in my Twitchs inventory, even though i received them prior (and the Miner is showing 0% on them). Something broke on Twitchs end 2 days ago.

RainOne commented 2 days ago

Yup, I'm getting the same results as gibbed - LastAwardedAt is messed up on my API call for campaign progress:

2:52:45: DEBUG: GQL Response: {'data': {'currentUser': {'id': '<useridgoeshere>', 'inventory': {'dropCampaignsInProgress': {'all my other drops go here'},{'game': None, 'id': '12a5d059-2338-4eba-8842-0d302d0058b3_CUSTOM_ID_2449', 'imageURL': 'https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/2af18c1d-0f03-493a-b02e-3de7ecc792a3.png', 'isConnected': True, 'lastAwardedAt': '0001-01-01T00:00:00Z', 'name': 'Parrot Wings', 'requiredAccountLink': 'https://twitchdrops.projectwinter.co/linkaccounts', 
'totalCount': 1, '__typename': 'UserDropReward'},{{remaining drops go here'}}

It keeps happening with every game that I already got the drop but it's returning a messed up "LastAwardedAt". Adding it to the ignore list obviously works and moves on to a new game that I didn't get the drops for - rinse and repeat.

Edit: Screw it, here's the 2mb log from app opening to when it starts farming for the "Parrot Wings" item (or maybe it was the "Bedazzled Blazer", both are from Project Winter). I already have both items (and they both show the "lastAwardedAt" messed up date.

TDMlog.txt

Windows200000 commented 2 days ago 
 elif (
            # If there's no self edge available, we can use claimed_benefits to determine
            # (with pretty good certainty) if this drop has been claimed or not.
            # To do this, we check if the benefitEdges appear in claimed_benefits, and then
            # deref their "lastAwardedAt" timestamps into a list to check against.
            # If the benefits were claimed while the drop was active,
            # the drop has been claimed too.
            (
                dts := [
                    claimed_benefits[bid]
                    for benefit in self.benefits
                    if (bid := benefit.id) in claimed_benefits
                ]
            )
            and all(self.starts_at <= dt < self.ends_at for dt in dts)
        ):
            self.is_claimed = True
        self._precondition_drops: list[str] = [d["id"] for d in (data["preconditionDrops"] or [])]

I can probably assume old ones are claimed, but for me an overwatch campaign one is from 9 days ago while it started 4 days ago.

Windows200000 commented 2 days ago

I just removed the requirement for lastAwardedAt to be after the start time. (included in in-dev build)

What this means, is that any drop, that was supposedly awarded before it started now counts as claimed.

THIS COULD LEAD TO DROPS BEING MISSED!!!

I don't know enough about what that date is supposed to mean and why it is checked, so please keep an eye out for that and let me know.

gibbed commented 2 days ago

I think if totalCount >= entitlementLimit then you can reasonably assume the drop cannot be further acquired, ignoring lastAwardedAt entirely.

Windows200000 commented 1 day ago

@gibbed > I think if totalCount >= entitlementLimit then you can reasonably assume the drop cannot be further acquired, ignoring lastAwardedAt entirely.

You might be right, but from what I understand, the inventory request, which is where claimed_benefits comes from, should already only include claimed drops. The use of timestamps, seems to have something to do with repeating drops, which I know no example of. I am thus unsure whether totalCount refers to amount claimed, or is just the same as benefit entitlementLimit.

Do you know some drops that can be repeated, that I could test with?

gibbed commented 1 day ago

Do you know some drops that can be repeated, that I could test with?

None that are currently active I think, I've seen repeat drops happen for Warframe but they haven't done that in awhile.

gibbed commented 1 day ago

Actually, I had a thought, and it was correct: totalCount absolutely refers to the total claimed and cannot correlate to entitlementLimit.

Here's a drop from NARAKA:

                        {
                            "id": "f3635c73-2c78-11ef-964b-0a58a9feac02",
                            "requiredSubs": 0,
                            "benefitEdges": [
                                {
                                    "benefit": {
                                        "id": "417c83f8-80bc-400e-9cde-0c3289ee0a47_CUSTOM_ID_9132752",
                                        "createdAt": "2024-04-03T09:42:37.305Z",
                                        "entitlementLimit": 1,
                                        "game": {
                                            "id": "515474",
                                            "name": "NARAKA: BLADEPOINT",
                                            "__typename": "Game"
                                        },
                                        "imageAssetURL": "https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/55e319d0-c8fa-43d8-ac99-0b63cac37aed.png",
                                        "isIosAvailable": false,
                                        "name": "Twitch Drops",
                                        "ownerOrganization": {
                                            "id": "417c83f8-80bc-400e-9cde-0c3289ee0a47",
                                            "name": "NetEase",
                                            "__typename": "Organization"
                                        },
                                        "__typename": "DropBenefit"
                                    },
                                    "entitlementLimit": 1,
                                    "__typename": "DropBenefitEdge"
                                }
                            ],
                            "endAt": "2024-07-01T18:59:59.999Z",
                            "name": "1",
                            "preconditionDrops": null,
                            "requiredMinutesWatched": 240,
                            "startAt": "2024-06-24T19:00:00Z",
                            "__typename": "TimeBasedDrop"
                        }
                        {
                            "game": null,
                            "id": "417c83f8-80bc-400e-9cde-0c3289ee0a47_CUSTOM_ID_9132752",
                            "imageURL": "https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/55e319d0-c8fa-43d8-ac99-0b63cac37aed.png",
                            "isConnected": true,
                            "lastAwardedAt": "2024-06-25T01:15:07Z",
                            "name": "Twitch Drops",
                            "requiredAccountLink": "https://www.narakathegame.com/news/guide/20230710/35647_1097721.html",
                            "totalCount": 8,
                            "__typename": "UserDropReward"
                        },

My assumption then is drops are tracked by that initial ID and not the entitlement list (ie, f3635c73-2c78-11ef-964b-0a58a9feac02).

Windows200000 commented 1 day ago

@gibbed I don't want to add a new way of handling stuff and just hope it works, before I have a way of testing it.

What you sent shows that entitlementLimit is still one, even if it is possible to claim it 8 times, I then don't know how progress would be tracked.

It is just a part of Twitch Drops I am not at all familiar with, so I can't include it before I find some current drops, ideally repeating more often, that I can use to figure out how that works.

Again, outside of repeating drops, the time check does nothing. The point of it is to somehow avoid marking repeating drops as completed. The above-mentioned commit implementing this is really old (basically from the first couple months of TDM) and I could not find a corresponding issue in the original repo I could use to learn more.

I am also afraid, that the current "fix" blocks repeating drops. I just need a campaign to test on and even then I'm not sure I'll have the time for like 3 more weeks. (#127)

uniquoo commented 1 day ago

Example from my side (Palia). I've already claimed all drops. Miner continues to farm them:

{
                        'game': None,
                        'id': 'a752483b-2886-11ef-ad15-0a58a9feac02',
                        'imageURL': 'https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/b581567a-59f4-4891-a069-1bf2dd51c2ab.png',
                        'isConnected': True,
                        'lastAwardedAt': '0001-01-01T00:00:00Z',
                        'name': '2 Palcat Pillow Plush',
                        'requiredAccountLink': 'https://accounts.palia.com',
                        'totalCount': 1,
                        '__typename': 'UserDropReward'
                    },
...
{
                        'game': None,
                        'id': '5fdd5148-baee-11ee-a640-0a58a9feac02',
                        'imageURL': 'https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/68dc33e7-c395-4c32-96c5-2557a7813e2d.png',
                        'isConnected': True,
                        'lastAwardedAt': '2024-02-10T11:23:19Z',
                        'name': '500 Lucky Envelopes',
                        'requiredAccountLink': 'https://accounts.palia.com',
                        'totalCount': 2,
                        '__typename': 'UserDropReward'
                    },
...
{
                        'game': None,
                        'id': '68f41f4e-28ed-11ef-9a9e-0a58a9feac02',
                        'imageURL': 'https://static-cdn.jtvnw.net/twitch-quests-assets/REWARD/0dc09056-a795-4c41-974d-b9e3a2612811.png',
                        'isConnected': True,
                        'lastAwardedAt': '0001-01-01T00:00:00Z',
                        'name': '1 Palcat Party Small Rug ',
                        'requiredAccountLink': 'https://accounts.palia.com',
                        'totalCount': 1,
                        '__typename': 'UserDropReward'
                    },
...
{
                    'id': '214dff39-2885-11ef-8108-0a58a9feac02',
                    'name': 'Palia Summer Drops',
                    'owner': {
                        'id': 'a3c13f82-ce43-4f29-88c0-47962f47809d',
                        'name': 'Singularity 6',
                        '__typename': 'Organization'
                    },
                    'game': {
                        'id': '982017546',
                        'displayName': 'Palia',
                        'boxArtURL': 'https://static-cdn.jtvnw.net/ttv-boxart/982017546_IGDB-120x160.jpg',
                        '__typename': 'Game'
                    },
                    'status': 'ACTIVE',
                    'startAt': '2024-06-25T20:00:00Z',
                    'endAt': '2024-07-16T19:59:59.998Z',
                    'detailsURL': 'https://palia.com/news/181-twitch-drops',
                    'accountLinkURL': 'https://accounts.palia.com',
                    'self': {
                        'isAccountConnected': True,
                        '__typename': 'DropCampaignSelfEdge'
                    },
                    '__typename': 'DropCampaign'
                },
...

All of said drops are not shown in my Twitch inventory anymore since 2 days. I received them prior to this bug. Drops having

'lastAwardedAt': '0001-01-01T00:00:00Z',

will get refarmed for obvious reasons.

notNSANE commented 1 day ago

Again, outside of repeating drops, the time check does nothing. The point of it is to somehow avoid marking repeating drops as completed. The above-mentioned commit implementing this is really old (basically from the first couple months of TDM) and I could not find a corresponding issue in the original repo I could use to learn more.

I am also afraid, that the current "fix" blocks repeating drops. I just need a campaign to test on and even then I'm not sure I'll have the time for like 3 more weeks. (#127)

Repeating drops on a single stream or any time?

Windows200000 commented 1 day ago

Again, outside of repeating drops, the time check does nothing. The point of it is to somehow avoid marking repeating drops as completed. The above-mentioned commit implementing this is really old (basically from the first couple months of TDM) and I could not find a corresponding issue in the original repo I could use to learn more. I am also afraid, that the current "fix" blocks repeating drops. I just need a campaign to test on and even then I'm not sure I'll have the time for like 3 more weeks. (#127)

Repeating drops on a single stream or any time?

I have no idea how these work.

Windows200000 commented 1 day ago

@uniquoo there is an in-dev release, that should have this fixed, but might cause drops that can be claimed multiple times to not be mined again.

uniquoo commented 1 day ago

@uniquoo there is an in-dev release, that should have this fixed, but might cause drops that can be claimed multiple times to not be mined again.

You said you dont really want to alter the code in regards of how it functions right? Would you want me to open a pull requests regardless so i can patch the function and change the check besides just removing lastAwardedAt , so repeatable drops will be possible again?

Windows200000 commented 1 day ago

@uniquoo there is an in-dev release, that should have this fixed, but might cause drops that can be claimed multiple times to not be mined again.

You said you dont really want to alter the code in regards of how it functions right? Would you want me to open a pull requests regardless so i can patch the function and change the check besides just removing lastAwardedAt , so repeatable drops will be possible again?

@uniquoo Yeah. I don't have much time, but I can review the pull request. Since my quick patch probably breaks repeat drops anyway, I'll be happy to merge anything that works for normal drops and see if people find issues.

edit: Make sure to create the pull request to in-dev.

notNSANE commented 1 day ago

Again, outside of repeating drops, the time check does nothing. The point of it is to somehow avoid marking repeating drops as completed. The above-mentioned commit implementing this is really old (basically from the first couple months of TDM) and I could not find a corresponding issue in the original repo I could use to learn more. I am also afraid, that the current "fix" blocks repeating drops. I just need a campaign to test on and even then I'm not sure I'll have the time for like 3 more weeks. (#127)

Repeating drops on a single stream or any time?

I have no idea how these work.

well, Smite repeats the same 7 or 8 drops every week. Marbles on Stream too.

If it's on the same stream, it barely happens. Perhaps Warframe, or Hearthstone, but these are very very rare.

Totesfleisch commented 23 hours ago

Shakes&Fidget repeat the same Drops all the Time, and Halo Infinite always offers the same Items, allthough in different Variables. I removed that Game from my Prio List after having every one of their drops atleast 10 times now, they usually are almost every Night.

barbiehunter commented 21 hours ago

Not 100% sure if this have something to do with this report too, but i just got this error (latest test build)

15:28:11: ERROR: Exception in task
15:28:11: Traceback (most recent call last):
15:28:11:   File "utils.py", line 139, in wrapper
15:28:11:   File "twitch.py", line 1398, in process_drops
15:28:11: asyncio.exceptions.InvalidStateError: invalid state