Looping & Resetting Progress (2023 years ago Twitch bug)

[iXerxes]

I assume this may be a Twitch bug, so there might not be a lot you're able to do here, however, I'm reporting it anyway, as it may be able to be worked around.

Recently, a lot of my drops appear as though they aren't being mined, specifically for me, the Overwatch 2 ones. The miner will watch a channel, progress for some hours and then the timer/progress will reset to zero if the channel is switched or something refreshes the miner.

After some looking around, it turns out I have already earned these drops, however, in my Twitch inventory page, the drops are marked as "Earned 2,023 years ago", and so they appear at the bottom of the page instead of at the top where the recently earned rewards appear.

I think this may be the reason the miner is constantly trying to farm the same drop over and over, as the drops that do this are the ones that show as being "Earned 2,023 years ago". For now, I will have to pay attention to which drops I have earned and exclude the game, but it would be nice if there was some work-around for this.

Thank you :)

[comradesean]

Confirming this is also happening to me although I was still using that fork fix up until today. Looks like it's another twitch change in the background.

[DevilXD]

Hello. Yeah, this definitely looks like some sort of a Twitch bug / visual issue being present. On my side, my last drop says it's been claimed 13 hours ago, and can be found at the top, so everything seems to be working just fine.

Are you using the latest dev version? Please note that I don't/cannot provide much support to you, if you're using a fork instead.

[iXerxes]

I was actually mining drops on three different accounts, and all three have the same issue. I was using the fork at that time, and when I started noticing things were off, I went back to your version (latest dev version). Same thing happens no matter which one I use

I don't have a very technical understanding of how the miner works, but is it possible that the miner caused this bug?

[matarife123]

false progress again, in palia and overwatch 2

[DevilXD]

is it possible that the miner caused this bug?

I was actually mining drops on three different accounts

@iXerxes Multi-account mining is not supported. The issues you're experiencing are most likely caused by Twitch rate-limiting the miner, which causes it to keep artificially increasing the progress, without doing any of the actual mining like it should be.

@matarife123 Are you sure it's the same Twitch account that's logged in the browser and the miner? Because your picture shows that it doesn't seem like it is, and you're looking at the wrong account, or something like that.

Again, just to be clear - for the miner to work, you cannot use it on more than one account at a time, and cannot watch any streams in your browser during the mining process. Otherwise, you'll keep getting false progress and it'll generally not work like you expect.

[matarife123]

¿Es posible que el minero haya causado este error?

En realidad, estaba extrayendo drops en tres cuentas diferentes.

@iXerxesNo se admite la minería multicuenta. Los problemas que estás experimentando probablemente se deben a que Twitch limita la velocidad del minero, lo que hace que siga aumentando artificialmente el progreso, sin realizar ninguna minería real como debería ser.

@matarife123¿Estás seguro de que es la misma cuenta de Twitch que está conectada en el navegador y en el minero? Porque tu imagen muestra que no parece ser así y que estás viendo la cuenta equivocada o algo así.

Nuevamente, para que quede claro: para que el minero funcione, no puedes usarlo en más de una cuenta a la vez y no puedes ver ninguna transmisión en tu navegador durante el proceso de minería. De lo contrario, seguirás obteniendo información falsa sobre el progreso y, en general, no funcionará como esperas.

It is my only Twitch account and I am not on another channel, right now it gave me other errors 22:30:00: ERROR: Broadcast settings update for a non-existing channel: 458358256 my last drop was 8 hours ago

[DevilXD]

Broadcast settings update for a non-existing channel

That's related to #110, which has been reported by you already a while ago. I need to finally get around to improving the websocket topics handling.

[matarife123]

I have restarted the miner, it says that palia has 2.5% progress but in my Twitch inventory there is no information about the drop. Same for overwatch2

[iXerxes]

@DevilXD I'm aware it's not supported for multiple accounts, however I have been using it like this for a while now, and it's always worked perfectly fine, up until the slight hiccups with the recent Twitch changes. Even still, I have been mining and getting drops on all accounts as expected.

When I close all instances and leave just one open, the issue still persists. I do actually have the drops for the affected games (2,023 years ago bug), but the miner doesn't seem to detect that, and keeps trying to make some kind of invalid/false progress.

Adding these games to the exclude list fixes the issue - for now - and allows the miner to make progress on the unaffected games, and does so just fine for all account instances running.

• I should note: I'm not running it on Windows. I'm using Ubuntu server with Xpra to have a virtual and persistent GUI which I connect to through a browser.

[iXerxes]

I have restarted the miner, it says that palia has 2.5% progress but in my Twitch inventory there is no information about the drop. Same for overwatch2

@matarife123 I recommend you go to the Twitch Inventory page and scroll to the bottom and see if your drops are there. As I've mentioned previously in this issue, some of the drops are marked as being as earned 2,023 years ago, and so they appear at the bottom of the page. For what ever reason, this causes problems for the miner. Also check in game for your drops just to be sure.

For now, you will have to add the affected games to the exclude list in the miner settings, and pay attention to when there are new drops so that you can un-exclude them.

[DevilXD]

I have been using it like this for a while now, and it's always worked perfectly fine, up until the slight hiccups with the recent Twitch changes.

The thing is, these "recent Twitch changes" have made it so that the exact same pathway as for downloading the stream content to watch it has to be followed. It's likely that Twitch has an already-implemented set of ways to detect a single IP address trying to watch 3 streams at once there, which causes what you're experiencing. That, or it's simply Twitch testing something, and you're getting caught in the process - gradual changes rollouts usually happen to only a selected subset of accounts at first, where it continues to grow as they decide to keep rolling out the update to everyone.

You can't simply downplay it with "but it worked previously", because the previous way of it working literally does not exist anymore. It's an entirely different pathway now, and as I said, it very likely uses the already-existing rate-limiting system Twitch most likely has.

When I close all instances and leave just one open, the issue still persists.

I recommend closing all instances for at least an hour, before concluding that anything "persists", as rate limits can get quite lengthy. A full 24 hours of not doing any mining on any account will let you rule out rate-limiting as the cause, and instead it'll mean it's just Twitch being buggy for you for some reason. Otherwise, we'll never know the real cause.

[iXerxes]

I recommend closing all instances for at least an hour

I have done this. I closed all instances after finding the bug and forgot about it for a day or two. After which, I loaded up a single instance and the issue still occurred. After adding the affected games to the exclude list, the miner works as expected for the other games.

[matarife123]

He reiniciado el minero, dice que palia tiene un 2.5% de progreso pero en mi inventario de Twitch no hay información sobre la caída. Lo mismo para overwatch2

@matarife123Te recomiendo que vayas a la página de Inventario de Twitch , te desplaces hasta el final y veas si tus gotas están allí. Como mencioné anteriormente en esta edición, algunas de las entregas están marcadas como obtenidas hace 2023 años, por lo que aparecen en la parte inferior de la página. Por alguna razón, esto causa problemas al minero. También revisa el juego para ver tus lanzamientos solo para estar seguro.

Por ahora, tendrás que agregar los juegos afectados a la lista de exclusión en la configuración del minero y prestar atención a cuándo hay nuevas caídas para que puedas anular su exclusión.

WTF is real, thanks. Right now I'm mining payday3 without problems

[DevilXD]

I have no real solution to this then, as it appears to be a Twitch-side bug, and I have no way to reproduce this on my side. I guess this issue can remain open in an off-chance someone else runs into this as well, but other than that, excluding the affected games may help solve the issue for you, and other than that, not much can be done.

I could possibly try to obtain the API response from you, to try and understand what causes the miner to try mining these drops again, but there's currently no way for the miner to save these. I should probably add it as a debug-feature of sorts. I'll try getting to it tomorrow.

[matarife123]

I have no real solution to this then, as it appears to be a Twitch-side bug, and I have no way to reproduce this on my side. I guess this issue can remain open in an off-chance someone else runs into this as well, but other than that, excluding the affected games may help solve the issue for you, and other than that, not much can be done.

I could possibly try to obtain the API response from you, to try and understand what causes the miner to try mining these drops again, but there's currently no way for the miner to save these. I should probably add it as a debug-feature of sorts. I'll try getting to it tomorrow.

I think Twitch changed the privacy policy several times, I could see the progress in the profile section like before but it seems like they removed it again. Also thank you very much for your effort and for the great work. ^^

[DevilXD]

https://github.com/DevilXD/TwitchDropsMiner/commit/f5d5fc3a05b949e2bbec2ef2b79940c30b614fac implements a new launch argument: --dump. Using it will launch the miner in a special mode, where it'll create a dump.dat file containing inventory data the miner uses to build it's internal campaigns "database", and then it'll close afterwards. It should be safe to share this file online, as it doesn't contain any user-related data, besides twitch account connection status to a particular game/campaign (connected or not), and current drop progress as shown on the inventory page.

If you want me to take a look, please attach the dump file here.

[matarife123]

I have added palia to the priority list and I have the same problem, it does not recognize that I have the drops

[DevilXD]

I'll need the dump file to try diagnosing it and provide any further help.

[iXerxes]

@DevilXD Here is my dump.

Currently, the these campaigns are/were bugged for me in some way:

• XDefiant Preseason - W6 (This drop is the most relevant to the issue.)

  • Weapon XP Booster - W6
  • Amber M44 Skin
  • Pelagic Vector Skin

  I have already completed this campaign, owning all of the drops, however, they all appear in my Twitch Inventory as being earned 2,023 years ago and the miner thinks I don't own them. The miner is currently trying to mine the first drop ("Weapon XP Booster - W6"), but any form of refreshing causes the progress to reset to 3.3%.

• PAYDAY 3

  I had no progress towards this campaign and the miner initially couldn't start to make any progress; also resetting to 3.3% upon refresh. After testing to see which of my campaigns were bugged, I was able to start progress on Twitch first, and now the miner is able to progress the drop as normal.

Campaigns that I have confirmed to be making valid progress (during this dump):

• Marbles on Stream
• Bloodhunt Summer Drops 1

Note:

For what ever reason, Marbles on Stream exported my Twitch UserID. I've replaced this ID with *MYUSERID*.

https://pastebin.com/xnizP6Va (Before PAYDAY 3 "Jumpstart") https://pastebin.com/N5KuzzEg (After PAYDAY 3 "Jumpstart")

[iXerxes]

Looking at the dump myself, it really doesn't show a whole lot to be honest. XDefiant just shows that I don't have the drops, and I haven't started the campaign, given that it doesn't have the self property.

PAYDAY 3 also tells me nothing, other than I hadn't yet started the drop in dump 1, and then I have progress in dump 2.

[DevilXD]

The most annoying thing about the self edge, is that it's not included when a campaign wasn't started yet, but it's also not included if a drop has already been claimed. And the information about it being claimed is normally in the self edge... So the miner does a little bit of "dancing around" with the info Twitch returns, to figure out if the drop has been claimed or not. One of those dances checks if the claim date on the drop lands within the timeframe of a campaign being active, and if so, it marks it as claimed. If not, it's considered unclaimed, and without a self edge, the progress on it becomes 0%, making it a valid mining target. There's also a catch of benefits vs drops to take care of there somewhere.

If the drop claim date is returned as 2023 years ago, then it's probably been marked as unclaimed, and that's why the miner is trying to mine it. I'll take a look at the dumps later today, once I get out of work.

[DevilXD]

I've taken a look at the dump file.

For what ever reason, Marbles on Stream exported my Twitch UserID

That's because it was included in the drop instance ID, which is a special ID that's sent to Twitch during drop claiming operation, and is the only way to tell it what you're claiming. If you have a campaign present in your inventory, that has at least one claimed (or ready to claim) drop, but not all of them are claimed yet, then the API will return self edges like that, including the drop instance ID that was used to claim that drop. I've totally forgot that's something that can include user IDs, but just pushed out a commit that ensures those are excluded from the dump: https://github.com/DevilXD/TwitchDropsMiner/commit/da050828c1e054a4197ca57568c78dbd0d1be6bc

As far as inspecting the dump goes, you have 22 drop instances that "were claimed" on the 1st of January 1 A.D, at midnight (0001-01-01 00:00:00+00:00), which is obvious nonsense and missing information. Without this claim date, it's not entirely possible to attribute the drops to any particular campaign, and whatever logic is assumed, it'll always be wrong in some cases:

• If we assume that wrong date == always claimed, that'll fix this issue, but introduce a new one where two consecutive campaigns for the same game, having the same benefit in each drop (like Smite, where it's a single benefit repeated x7, reappearing over and over in a new campaign every week), will get marked as finished (all drops claimed), even though the 2nd campaign hasn't been started yet. A user can fix this by manually watching at least one minute of an applicable stream (that'll put the unstarted campaign into inventory, and provide corrent claim status and progress), but it's no longer automatic and will mislabel drops. Basically #418.
• If we assume that wrong date == unclaimed, you get what we have here now - the miner trying to mine a campaign that was already mined and claimed. The only way to get around it, is to use the exclude list for the affected campaign, until it expires and can no longer be mined. This may also make you miss out on other campaigns for the same game.

Which one would you like to go with?

There's only one Payday 3 campaign, and it appears you have 15 minutes of progres on it already. There doesn't seem to be any other Payday 3 campaign present in the dump. All other values are as expected in the "before" dump as well. The only reason it wouldn't start on it's own, is the priority list focusing on something else, or no available channels to watch - otherwise it should start normally. If it didn't, then I have no idea why could that be.

[matarife123]

Could this have something to do with it?

[DevilXD]

@matarife123 No, that's just the drop that's currently being mined. This same drop is returned by GQL to check on mining progress, when the websocket decides to not respond for a while. It should generally be the same drop the websocket returns.

[iXerxes]

The only reason it wouldn't start on it's own, is the priority list focusing on something else, or no available channels to watch - otherwise it should start normally. If it didn't, then I have no idea why could that be.

Yea, I assumed this would be something the dump wouldn't be able to show. Sadly though, I was specifically testing to see which of my drops were working, and had all other available drops in the exclude list to isolate Payday 3. There were definitely people online and streaming it. Everything, from trying to test Payday in the miner, to jumpstarting the drop on Twitch its self, and back again - it all happened within the same 15-20 minutes.

Which one would you like to go with?

lol yea, idk either. These Twitch changes over the past several months have been a killer. I have actually emailed Twitch, however, showing them the glitched "Earned 2,023 years ago" drops, and they have said they're aware of it and are working on some solution. So hopefully this is something you can sit out, and just let it be until it passes :)

[DevilXD]

There were definitely people online and streaming it.

Was the miner watching a Payday 3 stream though? Or it didn't even attempt to watch it/watched something else?

[iXerxes]

Was the miner watching a Payday 3 stream though? Or it didn't even attempt to watch it/watched something else?

Yes, it was watching Payday 3. I even had to check a few times because there was some visual glitch where it looked like it was farming another game's drop, before some graphical update. So yes, I'm certain it was watching and attempting to progress Payday 3 - and failing to do so, before the "Jumpstart".

[DevilXD]

there was some visual glitch where it looked like it was farming another game's drop, before some graphical update.

That actually leads me to believe the miner was trying to do something else, or at least it's internal state lead it to believe it was trying to mine something else, other than Payday 3 campaign.

I'll put some more time into studying the dump file, in hope of finding an answer to this, but for now, it doesn't seem like it's something I can fix in any way. If Payday 3 stream was being watched, the campaign should start one way or another, regardless of what drop progress was being displayed.

[notNSANE]

this is happening again.

[DevilXD]

That's not enough information for me to even considered pursuing that further for now. Please provide a more extensive description of the exact issue you're experiencing.

[DevilXD]

Is this issue still happening to anyone here?

[DevilXD]

Since there has been no further reports on this, I consider this issue to be over.