How does the simulation work?

[Foxius]

I want to fork your repository and ran into the following problem: After I select a streamer to watch and start sending HEAD requests to the video player (as it was in your code), when checking the drop progress (based on DropCurrentSession), it outputs the following:

[2024-11-09 08:08:53,172] - TwitchSimulator - DEBUG -GQL Current Drop Response: [{'data': {'currentUser': {'id': '967038334', 'dropCurrentSession': {'channel': None, 'game': None, 'currentMinutesWatched': 0, 'requiredMinutesWatched': 0, 'dropID': '', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'durationMilliseconds': 57, 'operationName': 'DropCurrentSessionContext', 'requestID': '01JC7EAMEHGHS0RHHC4F8TXADS'}}]

That is, it does not count (as I understand it) viewing. Can you help with this? I'm not asking you to write code, I just want to know: what else besides head queries do you use to simulate watching?

[Foxius]

I forgot to clarify - if I go to the stream MYSELF (+- for a minute) before starting the script and then run the script, then it will work correctly

[2024-11-09 08:08:53,180] - TwitchSimulator - DEBUGGING - [OAuth 75a7] GQL's current response to deletion is: [{'data': _BOS_'Current user': _BOS_' ID': '975208803', 'dropCurrentSession': _BOS_' channel': _BOS_'id': '418513309', 'name': 'cok1es', 'Display name': 'Cok1es', "__typename": "Channel"}, "game": _BOS_"ID": "516866", "Display name": "STALCRAFT: X", "__typename": "Game"}, "Current views in minutes": 600, "Required views in minutes": 630, 'dropID': '6e82a343-96ad-11ef-b7bd-0a58a9feac02', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'Duration of milliseconds': 87, 'Operation name': 'DropCurrentSessionContext', 'RequestId': '01JC7EAMDWSRCRPGSSQB81CDA4'}}]

[DevilXD]

Hello,

I want to fork your repository and ran into the following problem:

[2024-11-09 08:08:53,172] - TwitchSimulator

I'm sorry, but I don't understand the problem here. This application isn't named "TwitchSimulator", and it doesn't look like anything my code would print out. "Forking" a repository involves starting with the existing code at base. If you're changing the existing code, you're kinda on your own when it comes to writing, using and debugging it.

However, if you're making something of your own, that works similar to the miner, then all I can tell you, is that Twitch tends to be buggy in how the progress is being reported. Expect it to not be reported at all, the response being complete nonsense (what you've got), the response looking okay, but pointing to a drop that you're not currently actually mining right now, the drop ID being correct but the rest of the response not making any sense, or actually getting something that looks like a good response and progress. In my miner, for the DropCurrentSessionContext operation and response, if dropID in the returned response doesn't point towards any drop that's currently tracked by the miner, or points towards a drop that cannot be mined on the currently watched channel, then the miner falls back to "pretend mining" mode, where the most likely drop to be mined has their progress increased. That's it.

[Foxius]

Yes. I didn't say it right, sorry. I'm not making a fork, I'm making my own miner based on yours. I understand correctly - if it is not possible to get the number of minutes that are left until the end of viewing, it looks at how many minutes are needed and outputs itself? And in principle I would like to understand more about “pretend mining” mode.... Also, as I understand, it counts minutes but just does not show them?

[Foxius]

i tested it now. My results:

5 minutes sending head requests and get that response from DropCurrentSessionContext:

[2024-11-10 20:34:13,863] - TwitchSimulator - DEBUG - [OAuth g078] GQL Current Drop Response: [{'data': {'currentUser': {'id': '775030133', 'dropCurrentSession': {'channel': None, 'game': None, 'currentMinutesWatched': 0, 'requiredMinutesWatched': 0, 'dropID': '', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'durationMilliseconds': 58, 'operationName': 'DropCurrentSessionContext', 'requestID': '01JCBBC5DBHFGJ66715T34HHND'}}]

i join to stream nmplol and see this in my console:

[2024-11-10 20:34:55,630] - TwitchSimulator - DEBUG - [OAuth g078] GQL Current Drop Response: [{'data': {'currentUser': {'id': '775030133', 'dropCurrentSession': {'channel': {'id': '21841789', 'name': 'nmplol', 'displayName': 'Nmplol', '__typename': 'Channel'}, 'game': {'id': '263490', 'displayName': 'Rust', '__typename': 'Game'}, 'currentMinutesWatched': 1, 'requiredMinutesWatched': 60, 'dropID': 'fe9601ff-9d1b-11ef-a7ae-0a58a9feac02', '__typename': 'DropCurrentSession'}, '__typename': 'User'}}, 'extensions': {'durationMilliseconds': 73, 'operationName': 'DropCurrentSessionContext', 'requestID': '01JCBBDE6JYJG2CKP1ZD9WDRX5'}}]

So he didn't count the five minutes of viewing time until I logged on to the stream :(

[DevilXD]

And in principle I would like to understand more about “pretend mining” mode.... Also, as I understand, it counts minutes but just does not show them?

The part of the code responsible for that can be found here: https://github.com/DevilXD/TwitchDropsMiner/blob/d4cfcb5d52412395dd14eb5fc5420552230c2a21/twitch.py#L922-L934

get_active_drop just iterates over all campaigns that are tracked, and calls can_earn(watching_channel) on them. If a campaign can be earned on the currently watched channel (game matches, ACL matches or not present, etc.) All drops that can be earned from such campaigns are put into a list of options to choose from. The miner then sorts the list and outputs the drop that has the least amount of remaining minutes left. That drop then has it's progress "artificially" increased by 1 minute. That's about it. Most of the time, these theoretical progress increases match the progress you can see on the website. The progress will "resync" back to it's true value on the next inventory refresh, or when Twitch decides to actually return proper progress in one of the later responses.

5 minutes sending head requests So he didn't count the five minutes of viewing time

There's actually a 5 minutes period of "uncertainty" that I've noticed happens on Twitch side, each and every time you deviate from steady mining on a single channel. The progress is misreported during that period the most, and is the main reason the "pretend mining" mode even exists. The period starts every time you start mining a particular channel, so it also happens after channel switches.

After those 5 minutes of steadily mining the channel pass, the progress reporting goes more or less back to normal. You have the option to either ignore it, or mask it with "pretend mining" like I did.

As long as the progress keeps increasing as reported by the inventory page, you're doing everything right, even without proper progress reporting.

[Foxius]

What if it doesn't show up in my inventory either? I logged into a new account and sent requests to view it. the inventory was empty. Here's the code I'm using (it's test code, not from the project I threw above, but it's literally copied into the project so they work the same way).

import asyncio
import aiohttp
from typing import Dict
import logging
from exceptions import GQLException, MinerException

logger = logging.getLogger("TwitchDropsSimulator")
logger.setLevel(logging.DEBUG)

console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)

logger.addHandler(console_handler)

import requests

class TwitchSimulator:
    def __init__(self, oauth_token: str, username: str):
        self.oauth_token = oauth_token
        self.username = username
        self.session: aiohttp.ClientSession | None = None
        self.api_url = "https://gql.twitch.tv/gql"

    async def connect(self):
        self.session = aiohttp.ClientSession(headers={
                "Authorization": f"OAuth {self.oauth_token}",
                "Client-ID": "kimne78kx3ncx6brgo4mv6wki5h1ko",
            })

    async def close(self):
        if self.session:
            await self.session.close()

    async def gql_request(self, data: str) -> Dict:
        async with self.session.post(self.api_url, data=data) as response:
            if response.status != 200:
                logger.error(f"GQL request failed with status {response.status}")
                raise GQLException(f"GQL request failed with status {response.status}")
            data = await response.json()
            if "errors" in data:
                logger.error(f"GQL request returned errors: {data['errors']}")
                raise GQLException(f"GQL request returned errors: {data['errors']}")
            return data

    async def get_stream_url(self) -> str:
        data = '''[{
            "operationName": "PlaybackAccessToken_Template",
            "query": "query PlaybackAccessToken_Template($login: String!, $isLive: Boolean!, $vodID: ID!, $isVod: Boolean!, $playerType: String!, $platform: String!) {  streamPlaybackAccessToken(channelName: $login, params: {platform: $platform, playerBackend: \\"mediaplayer\\", playerType: $playerType}) @include(if: $isLive) {    value    signature   authorization { isForbidden forbiddenReasonCode }   __typename  }  videoPlaybackAccessToken(id: $vodID, params: {platform: $platform, playerBackend: \\"mediaplayer\\", playerType: $playerType}) @include(if: $isVod) {    value    signature   __typename  }}",
            "variables": {
                "isLive": true,
                "login": "''' + self.username + '''",
                "isVod": false,
                "vodID": "",
                "playerType": "site",
                "platform": "web"
            }
        }]'''

        print(data)
        response = await self.gql_request(data)
        logger.debug(f"GQL Stream URL Response: {response}")

        token_data = response[0]["data"]["streamPlaybackAccessToken"]
        playback_url = f'https://usher.ttvnw.net/api/channel/hls/{self.username}.m3u8?sig={token_data["signature"]}&token={token_data["value"]}'

        async with self.session.get(playback_url) as resp:
            m3u8_data = await resp.text()

        # Найти последний сегмент потока
        lines = m3u8_data.splitlines()
        for line in reversed(lines):
            if line.startswith("http"):
                return line

        raise MinerException("No stream segments found")

    async def simulate_watch(self):
        try:
            stream_segment_url = await self.get_stream_url()
            logger.info(f"Simulating watch on segment: {stream_segment_url}")

            while True:
                async with self.session.head(stream_segment_url) as resp:
                    if resp.status == 200:
                        logger.info("HEAD request successful, simulating watch.")
                    else:
                        logger.warning(f"HEAD request failed with status {resp.status}")
                await asyncio.sleep(20)  # Отправка HEAD запроса каждые 20 секунд
        except Exception as e:
            logger.error(f"Simulation error: {e}")

    async def get_drop_progress(self) -> Dict:
        data = f'''[{{"operationName":"DropCurrentSessionContext","variables":{{"channelLogin":"{self.username}"}}, "extensions":{{"persistedQuery":{{"version":1,"sha256Hash":"4d06b702d25d652afb9ef835d2a550031f1cf762b193523a92166f40ea3d142b"}}}}}}]'''

        response = await self.gql_request(data)
        logger.debug(f"GQL Current Drop Response: {response}")

        # drop_session = response[0].get("data", {}).get("currentUser", {}).get("dropCurrentSession", {})
        # if not drop_session:
            # raise MinerException("No active drop session available")

        # return drop_session

    async def run(self):
        try:
            await self.connect()

            stream_segment_url = await self.get_stream_url()
            logger.info(f"Simulating initial watch on segment: {stream_segment_url}")

            # for i in range(5):
                # async with self.session.head(stream_segment_url) as resp:
                    # if resp.status == 200:
                        # logger.info(f"HEAD request #{i + 1} successful.")
                    # else:
                        # logger.warning(f"HEAD request #{i + 1} failed with status {resp.status}")
                # await asyncio.sleep(20)

            # drop_progress = await self.get_drop_progress()
            # logger.info(f"Drop session data: {drop_progress}")

            # required_minutes = drop_progress.get("requiredMinutesWatched", 0)
            # current_minutes = drop_progress.get("currentMinutesWatched", 0)
            # remaining_minutes = required_minutes - current_minutes

            # logger.info(f"Remaining minutes to drop: {remaining_minutes}")

            # watch_task = asyncio.create_task(self.simulate_watch())
            # remaining_minutes = 123012321321092133213128231890231890
            while True:
                # logger.info(f"Remaining minutes: {remaining_minutes}")
                await asyncio.sleep(60)
                remaining_minutes -= 1

            watch_task.cancel()
            logger.info("Drop progress completed!")
        finally:
            await self.close()

if __name__ == "__main__":
    OAUTH_TOKEN = ""
    USERNAME = "nmplol"

    simulator = TwitchSimulator(OAUTH_TOKEN, USERNAME)
    asyncio.run(simulator.run())

[DevilXD]

What if it doesn't show up in my inventory either?

You can "prime" a drop to appear in the inventory, by watching a single minute of a stream. Not-started campaigns are only available on the campaigns page otherwise. Once it shows up in the inventory, you can track it until it gets finished and disappears off there.

I logged into a new account and sent requests to view it. the inventory was empty.

Well, you're doing something wrong then. I can't really help you develop the entire thing, my project took weeks (or even months) of research to arrive at the point it's currently at now. If you're not getting the progress, then it can be anything. The most troublesome issue I can see with the code you've posted, is using the Web version Client ID (starts with "kimne"), as the Twitch web client has been protected by the integrity system by Twitch themselves, in response to this and some other projects starting to automate drop acquisition. Using it usually won't lead you anywhere. I recommend trying again with either the SmartBox or Mobile client IDs instead.

[Foxius]

SmartBox или Mobile.

Can I find them in your code?

[DevilXD]

Yes. Most of the answers for your issues can be solved by just studying the existing code.