Twitch blocking proxies after some time

[Tobi4s1337]

I am experiencing a similar behavior as you described (twitch blocking proxies) even when using residential proxies. In my case, it works fine for around 2 minutes and then the viewers disappear. When using the same proxies again after restarting the bot the viewers don't appear at all.

It seems like twitch is using another way to make sure that it is a "real" user and not a bot, but so far I am not sure how they do it.

(Thank you for publishing your research, it helped me a lot to get started with a Nodejs viewbot)

[breuerfelix]

very interesting... i thought that it might work with residential but hmm that is bad!

one questions, does this message appear for you error getting stream url in thread {thread_id}. try streamlink ?
i have the feeling that when using streamlink url, it does not work because there are some cookies missing.
that would mean that twitch is already blocking the initial request to twitch.tv.

[Tobi4s1337]

So far I did not yet see the message you mentioned ( error getting stream url in thread {thread_id}. try streamlink ).

I wonder if there is something similar to a heartbeat request, which is there to prevent view botting? At least it is interesting to see that it works fine for like 1-2 minutes and then the IPs get blacklisted (most-likely).

[breuerfelix]

well just open the browser and watch the network tab. there is no heartbeat :)

BUT i only do head requests when requesting the actual stream. so maybe thats a problem.

you could try to change that and do an actual get request which produces way more traffic, but maybe this works :D

[Tobi4s1337]

Yeah, I checked it yesterday and there is no heartbeat or something similar. It seems like I was counted for a viewer even when blocking all requests that are not fetching the stream parts.

I reimplemented the auto watcher in my node app and so far it seems to work way better. Now the problem is not that my proxies are getting blocked fast but that they sometimes time out (so need to add something so it retries a couple of times and if the proxy times out every time, it should start a new Thread with a different proxy). Is there a specific reason why you commented the auto watcher out?

[breuerfelix]

i commented the auto watcher out because the browser is only fetching the latest prefetch url anyways ... and the problem were the proxies for me, so i didnt wanted to have this complex algorithm i didnt know if it is working anyways :D

how have you implemented it in the node ? you made a auto watcher like i did ?
what differences do you have in your app instead of mine ?

cause thats impressive that it is working for you! i am curious on what i did wrong :D

also does it work with head or with get requests for you ?

[Tobi4s1337]

        getStreamURL(options, channel, token, (URL) => {
            console.log("starting something here")
            run = true
            let allBatchURLs = [];
            let sortedTime = [];
            let lastPlaylist = Date.now() - 100;
            let lastSegment;

            function watchSegment() {
                if (lastPlaylist <= Date.now() - 100) {
                    requestPlaylist(options, channel, URL, (videoURLs) => {
                        if (videoURLs) {
                            for (let i = 0; i < videoURLs.length; i++) {
                                let allBatchURLsLength = allBatchURLs.length;
                                if (allBatchURLs.length > 0) {
                                    for (let n = 0; n < allBatchURLsLength; n++) {
                                        if (allBatchURLs[n].timestamp == videoURLs[i].timestamp) {
                                            allBatchURLs[n].link = videoURLs[i].link
                                            allBatchURLs[n].toRemove = false;
                                            break;
                                        } else if (n + 1 === allBatchURLs.length) {
                                            allBatchURLs.push(videoURLs[i])
                                        }
                                    }
                                } else {
                                    allBatchURLs.push(videoURLs[i])
                                }
                            }
                            sortedTime = allBatchURLs.sort(compare);
                        }
                        if (lastSegment) {
                            // mark items to remove
                            for (let i = 0; i < sortedTime.length; i++) {
                                if (sortedTime[i].timestamp < lastSegment) {
                                    sortedTime[i].toRemove = true;
                                }
                            }
                            // remove items
                            sortedTime = sortedTime.filter(obj => obj.toRemove === true);
                        }
                        lastPlaylist = Date.now()

                        let earliestBatchTime = ""
                        let batchURL = ""

                        if (sortedTime.length > 0) {
                            earliestBatchTime = sortedTime[0].timestamp;
                            batchURL = sortedTime[0].link;

                            sortedTime = sortedTime.filter(obj => obj.timestamp === sortedTime[0].timestamp);
                            allBatchURLs = allBatchURLs.filter(obj => obj.timestamp === earliestBatchTime)
                        }

                        lastSegment = earliestBatchTime
                        requestSegement(options, channel, batchURL)
                        setTimeout(watchSegment, 2000)
                    })
                }

            }

            watchSegment()

        })

I did actually just rewrite your python app using javascript, same for the auto-watcher. It is still not really working, but it does not blacklist the IPs anymore if I am correct.

Before without using the auto-watcher, it would work for 45 seconds, then the viewers would disappear. Now the viewers stay there for a couple of minutes in case they don't time out (currently the thread stops if there is one request that times out, so need to improve that). Eventually, all the viewers from my bot are gone because requests failed for each thread, BUT: I can just restart my node app and the viewers are back. Without using the auto-watcher, the viewers would not show up anymore, because they got backlisted (at least that is my theory).

What do you think? ( I am using HEAD requests )

Edit: Something else I do is that I create random user agents using a npm module. I am not sure if that makes a big difference.

[breuerfelix]

ahhh okay interesting :D so actually my approach with the auto watcher was not that bad :D

you know what also got me thinking ? imagine being in the university. all computers there got the same IP cause they use the same router. if 100 people watch streams, there will be 100 viewers even though its all from the same IP. how ?

MAYBE you dont even need to use proxies and just generate new user agents (the initial request to twitch.tv generates new cookies anyways). or to make it even safer, create different accounts and login and watch this these accounts from the same ip (which shouldnt get you into timeouts and which is NOT a problem because in the university you can also watch twitch from the same IP with like 100 different accounts without getting blacklisted right ?

/btw felixbreuer#9641 on discord :) if you got that

[Tobi4s1337]

I think Twitch has a limit at least for how many users will get counted as viewers when coming from the same IP ( old but good blog post about it I found: https://www.ericzhang.me/faking-views-on-twitch-tv/ ). He mentioned that up to 10 Viewers can use the same IP, but using my VPS I can't create more than one viewer, even when using different user agents =/ But who knows, maybe my VPS IP is already blacklisted because of the testing I did so far...

I'll add you on discord (Tobi4s_#8987)

[Twistions]

Viewers are disappearing after 2 minutes because of not proxies, you are missing something. And something more: twitch accepts max 2 viewers per ip maybe 3 i am not sure 100% about it.

[Tobi4s1337]

@kreekton do you have any idea what might be missing?

[ViolentScarlett]

Hey guys, interested in this piece of code. @breuerfelix did you incorporate the javascript from @Tobi4s1337 ? Also, does a VPN help on this?

[nick0zer]

hi there guys, the problem is not the proxies, twitch flag viewers that have the stream muted and count them as inactive viewers. ips that get flagged will not be able to be counted as viewers after a certain of time. also maximum viewers per ip is 10 if you can fix the code to not mute the video will probably fix the problem

[breuerfelix]

@nick0zer this is not true. if i watch my own stream and mute myself, i still get me as a viewer. also muting or unmuting the player triggers NO network request so twitch doesn't even know about you muting the player. ALSO video and audio do not stream separately. so twitch is for real not being able to know that.

have you tested the maximum amount of viewers per IP ? so if you open 10 incognito windows with your own stream you get 10 viewers ? i doubt that but i haven't tested so i cant verify that. /edit this does at least not work in private firefox windows. tried 4 windows, still only 1 view count also every single windows was muted. still got one view count. please do not spread myths without testing yourself.

[nick0zer]

it is actually true, twitch knows when the video player is muted manually and when its muted automatically. There are few viewbots that got this fixed by muting the entire process and not the video player

[nick0zer]

Btw, I've tested my 1k private proxies from fineproxys website and with StreamerTools view bot I managed to get 1500 viewers with one instance of the bot, and with some outdated twitchGod/rubot I was getting 700-800 viewers for 3 minutes than the viewers will disappear. Clearly its not about private commercial proxies being blocked by twitch but its the way you sent the viewers. Also yes the limit for a single IP is 10 viewers. Tested before some years I believe they haven't changed

[OGSegu]

Btw, I've tested my 1k private proxies from fineproxys website and with StreamerTools view bot I managed to get 1500 viewers with one instance of the bot, and with some outdated twitchGod/rubot I was getting 700-800 viewers for 3 minutes than the viewers will disappear. Clearly its not about private commercial proxies being blocked by twitch but its the way you sent the viewers. Also yes the limit for a single IP is 10 viewers. Tested before some years I believe they haven't changed

1. 10 viewers per single IP limit doesn't work for like 2 years.
2. As @breuerfelix said if the stream is muted there's no request to Twitch, it's clientside action.

[breuerfelix]

@OGSegu thanks for the support but i stopped trying to argue if the other person is clearly only using bots and not developing bots.

btw i got some more testing: if i watch a stream from my lokal computer with the streamlink library / cli the stream gets a viewer. if i use fast mobile proxies and watch the stream (not just head requests) with the streamlink library setting the proxy (since the library offers that), no viewers are added.

that means, at least i think so, that it is all about the IP. you need residential proxies or you are basically fucked. cause in that case it was not about the algorithm and not about "pretending" to watch. i really wanna know how all these bot services do that, i guess they just got good residential proxies.

[Twistions]

I am the main programmer of one of the online twitch bot websites. You have to use residental proxies and your requests have to be realestic so they won't be blocked. What do i mean with realestic is, there are some other requests imporant too, not only ttvnw.net requests. They are blocking ips if you use them suspiciously. About the muting video sound, twitch sends every 10 - 60 seconds a request which has a lot of information about what client does. Twitch gathering it statistically it doesn't block from the viewer count.

[breuerfelix]

@kreekton awesome! thanks for the clarification :) well i also found out about that 10-60 seconds request and blocked it from my firefox browser. Viewercount still appears so i thought it might not be important, and you say the same :) it is sort of performance measurement of their servers, at least thats what i get from the source code.

so you need more of the initial handshake than i am doing. i tried with selenium but even 10 viewers are too much for a computer... well i think the biggest problem for a casual python programmer are the residential proxies :D hard to get them!

is there a chance we can have a chat on discord ? felixbreuer#9641

[Twistions]

@kreekton awesome! thanks for the clarification :) well i also found out about that 10-60 seconds request and blocked it from my firefox browser. Viewercount still appears so i thought it might not be important, and you say the same :) it is sort of performance measurement of their servers, at least thats what i get from the source code.

so you need more of the initial handshake than i am doing. i tried with selenium but even 10 viewers are too much for a computer... well i think the biggest problem for a casual python programmer are the residential proxies :D hard to get them!

is there a chance we can have a chat on discord ? felixbreuer#9641

i have sent you a friend request, and do you block the navigator.webdriver javascript property? more information about this. https://stackoverflow.com/questions/53039551/selenium-webdriver-modifying-navigator-webdriver-flag-to-prevent-selenium-detec

[breuerfelix]

ye i did that when i tested selenium but my bot doesnt use selenium :) i replay all the requests without an actual browser :) but i programmed a firefox extension for instapy back in the days which sets ALL windows variables according to a real browser even though you are headless. https://github.com/timgrossmann/InstaPy/tree/master/instapy/firefox_extension

[OGSegu]

I am the main programmer of one of the online twitch bot websites. You have to use residental proxies and your requests have to be realestic so they won't be blocked. What do i mean with realestic is, there are some other requests imporant too, not only ttvnw.net requests. They are blocking ips if you use them suspiciously. About the muting video sound, twitch sends every 10 - 60 seconds a request which has a lot of information about what client does. Twitch gathering it statistically it doesn't block from the viewer count.

I have coded Twitch View Bot which is working on these "secret" requests. They can't block residential proxy, because it changes every request. The only thing that can stop you is that the request has a "Client-ID' param and it has some rate limit. Using selenium is bad practice right here because you need to care about a lot of things. You just can send an HTTP request if you can find the right ones :)

[WexLab]

Hi, you guys have some progress in writing code, I tried for 5 months to squeeze out more than 40 views, but then they stopped going at all. I couldn't find a solution to this problem. I can try to help you guys, if you want my discord wexlab#2636