ALL twitch.tv issues

[PhiZero]

URL(s) where the issue occurs

twitch.tv

Describe the issue

Embedded video ads are showing again after the fix from https://github.com/uBlockOrigin/uAssets/issues/5138 that lasted until now.

Screenshot(s)

Versions

• Browser/version: Chrome 73.0.3683.75
• uBlock Origin version: 1.18.0

Settings

Default settings.

Notes

Fix that was pushed as part of issue 5138 worked until a few hours ago, now getting embedded ads again. No changes made between then and now. Maybe a recent filter change broke this, or maybe Twitch patched this workaround?

[lain566]

The @pixeltris's solution has been broken.

[pixeltris]

The next logical step is removing the ad chunks in the stream. This unfortunately can add quite a delay until the actual stream starts. See https://github.com/instance01/Twitch-HLS-AdBlock

EDIT: If you play around with the params to the access_token request there seem to be more workarounds. Probably futile to do so though? Maybe implement something similar to Twitch-HLS-AdBlock and then add any additional workarounds as they are found?

[stoically]

I've just noticed that the initial request, when entering a stream, to https://usher.ttvnw.net/api/channel/hls/<channelname>.m3u8 (serves the playlists for the different encodings) contains a parameter called show_ads set to true. Might be worth trying to rewrite that one to false.

[TheEnhas]

The platforms (aside from the regular site) are listed here, but I'm not sure what the actual values to use are for each: https://www.twitch.tv/p/platforms/

Might be worth trying though. I think it's odd that platform being undefined worked (for a while) to prevent ads, which means that was probably the default and that the other values are also meant to enable them. On reddit someone (supposedly) doesn't get ads via FireTV so that one may work.

[pixeltris]

@stoically that request includes a "sig" param which looks like a SHA1 based on the length. The json request is probably hashed along with some server side secret so I don't think it's feasible to modify that request.

@TheEnhas there is at least one value which can currently bypass the server_ads thing and avoid ads. But I think with a great certainty if it were added to uBlockOrigin it would get fixed within 24 hours.

[TheEnhas]

@TheEnhas there is at least one value which can currently bypass the server_ads thing and avoid ads. But I think with a great certainty if it were added to uBlockOrigin it would get fixed within 24 hours.

If it's by design on some platforms to save bandwidth or that already have other ads then I'm not sure if they'd "fix it", but I get the feeling it's more likely that they've just not updated those apps yet to work with the new system and they'll get the ads like everyone else once they do.

[stoically]

@pixeltris Although you're probably right and the token needs to match the sig, I wanted to at least give it a try - but it seems that the overwritten fetch in twitch-videoad.js never gets called with the request to usher.ttvnw.net. Could the reason for that be that the call is maybe made inside a ServiceWorker? Is there a way for uBO to inject something into ServiceWorkers?

FWIW, I've also tried to redirect the request using the WebExtensions onBeforeRequest API, but that leads to a Network Error in the Twitch player and it also doesn't show the request at all in the Network log. The only reason I can think of that causes this, is that the fetch request.redirect for that request is set to manual and fails for non-200 status.

[pixeltris]

@stoically https://gist.github.com/pixeltris/e659867516ed0be149bf5e9e22d7d6e0 something like this should work (I have been using it for tests, modifying the response on a locally hosted server so I don't have to deal with the pain of javascript).

[stoically]

That works, thanks. Though rewriting the token params results in the server responding with a 403 error, so you're probably right about the sig.

[pijcab]

Hi guys, Is there a documentation on how to add the sort of snippet @pixeltris is giving us ? Im a complete noob when it comes to modifying Chrome/Firefox extensions... Edit : This Twitch ad thing is getting out of hand, if I'm a Twitch Prime user I dont want to watch ads on top of that (ads which are usually for services like Amazon Video that are meant for Prime subs...)

[stoically]

@pijcab Select advanced in the uBO dashboard, click the appearing cog, replace unset for userResourcesLocation with the raw url to the gist (needs a click on raw on the linked gist). This will break all rules which depend on the default resources that are not included in the gist.

[Jap2-0]

Wouldn't sig= be generated client-side, or no?

[stoically]

Yeah, it looks like the sig is generated in the WASM module. Call stack for the request (version 2.9.2):

[pijcab]

@stoically thank you very much, didn't know that userResourcesLocation was used for that.

Also this might help some people : I found a workaround by using a VPN and connecting to the countries outside of the SureStream available geos (Such as Romania or Hungary). Didn't get any ads for an hour or two as of now

[TheEnhas]

Some people reportedly had success by changing their browser's user agent to Android while on Twitch. I tried it and ads still got through.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/523c2fa60547046bbd15122e5aa70489f5d8afde/twitch_hls_test.js this is a rough copy/paste of @instance01's https://github.com/instance01/Twitch-HLS-AdBlock for uBlock Origin. Last time I tested his extension I was getting reasonable load times (~5 seconds). Currently everything seems to take like 30+ seconds to load. I haven't looked too much into it yet and had to gut out some code which was causing spam requests.

I tried making a "loading screen" .ts file but failed miserably. If anyone knows how to make a valid .ts file Twitch can load that would be useful. There is a base64 string where the .ts file should go at the bottom of that file. Otherwise the screen will be black for ~30 seconds while ads are displayed.

[TheEnhas]

Is there a value set anywhere that tells Twitch that you've seen an ad recently so as to not play another one? Because (one big thing I hate about the new pre-roll system) you end up behind the stream once the ad finishes and have to pause/play or reset the player to get back to the right latency, and you don't get another ad right away if you do that. Also, even if I reload or change streams after getting an ad I don't tend to get another for some time. This doesn't make a lot of sense because some people report getting an ad every time they load a stream no matter what, but in my case maybe it's setting some value somewhere that expires after like a half hour or whatever amount of stream loads (whichever comes first).

[stoically]

@TheEnhas The ads are at least geo-aware, if not targeted. So it's probably some server-side logic that decides whether to include ads in your m3u8 playlists or not (hence the name SSAI). Something like: Some IP-ranges (or targeted users if you're logged in) are limited to a Pre-roll Ad every 15minutes, others are not - based on the options chosen by advertisers.

Related or maybe even superseding issue: https://github.com/uBlockOrigin/uAssets/issues/1905

[CaelThunderwing]

likely Ad blocking may break for me on April 2nd (older Amazon Prime/Twitch Prime and i guess? i was grandfathered into having ads disabled as i have even uBlock disabled and still get squat but i disabled since i made affiliate as to not even tempt to breaking contract,) for when i haveto let it expire briefly to renew w/ a gift of prime... Anyway..

maybe look into how Kodi & the twitch plugin does this because no ads play before or during a stream. I have a media center box in my gaming room setup with Kodi 18.1 and latest twitch plugin outside a few edge cases that streams either crash the box or wont load(a kodi fault/plugin fault) i dont get ads whatso ever, with or without logging into my twitch account.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/0c9f3d988d9cd0ef5f011ad8057a3d50ae23a38c/twitch_hls_test.js updated version of the adapted Twitch-HLS-AdBlock for uBlock Origin. It should load the stream in 5-10 seconds now rather than 30. Not sure about midroll ads though. Actually maybe 10-30 seconds? Pretty bad still.

[TheEnhas]

Is there any way to get the time length of the ad at the start, immediately skip to the end of that length and mark it as being "viewed" by Twitch so that you end up with very little (if any) downtime?

[pixeltris]

@TheEnhas that's essentially what Twitch-HLS-AdBlock / the thing I linked are doing. There is one m3u8 per stream quality, as it polls the m3u8 file, normal stream segments will start showing up. As soon as they are available they are displayed (and all the ad segments removed). Unfortunately there is still quite a delay before the normal stream is available. Maybe you could spam the m3u8 some more but I doubt it would help.

[stoically]

Just realized that the token and sig in the /<channelname>.m3u8 request are the exact return values from the initial call to /access_token, so yeah, @pixeltris's assumption of server-side hashing is correct.

@pixeltris Regarding Pre-rolls, the m3u8 requests for me look like this when loading the stream:

• First Ad:
  • Request time: "Wed, 20 Mar 2019 14:48:47 GMT"
  • m3u8 value: #EXT-X-PROGRAM-DATE-TIME:2019-03-20T14:48:35.822Z
• First non-Ad (after SCTE35-IN):
  • Request time: "Wed, 20 Mar 2019 14:48:53 GMT"
  • m3u8 value: #EXT-X-PROGRAM-DATE-TIME:2019-03-20T14:48:49.822Z

So the Ad plays 14seconds but the first actual stream .ts is served after 6seconds. However, I think that the m3u8 also includes .ts segments which only will be made available in the future and are not directly available when included in the m3u8 (30-60seconds m3u8 segments vs 2-5seconds stream-delay). I guess the twitch player uses the #EXT-X-PROGRAM-DATE-TIME and that could somehow be rewritten accordingly for the first available non-Ad. Here's the raw HAR for the Pre-roll.

Stripping segments completely works for Pre-rolls, since the player didn't start yet and will start correctly with the first non-Ad segment - but it will break for Mid-rolls because the player already started and suddenly doesn't get segments anymore. Also it seems that Mid-rolls don't serve non-Ad segments early. (More information about my Mid-roll findings including attempts to create a placeholder .ts with ffmpeg are here)

Would be interesting to see other HAR's for Pre-rolls and Mid-rolls (here's how to obtain one: Firefox / Chrome).

[pixeltris]

I'm not sure you can completely strip the m3u8 of all segment files as otherwise the twitch player will start spamming the server hard until it finds something to play. That's why I inserted a dummy .ts entry when there are only ads (it would be ideal if this would be an actual .ts file which displays a message about ads being blocked). Does the current solution I linked break when midrolls are played? I haven't had a chance to see a stream that has played midrolls. Is there a streamer who plays them often?

[pijcab]

@pixeltris try Shroud

[stoically]

@pixeltris

I'm not sure you can completely strip the m3u8 of all segment files as otherwise the twitch player will start spamming the server hard until it finds something to play.

It seems as long as the m3u8 can get parsed (request result not empty), it will not result in the "Invalid variant" error and spam, but rather poll normally.

it would be ideal if this would be an actual .ts file which displays a message about ads being blocked

Did you try the linked ffmpeg command?

Does the current solution I linked break when midrolls are played? I haven't had a chance to see a stream that has played midrolls. Is there a streamer who plays them often?

Hadn't a chance to test with Mid-rolls yet. I know that /twitchpresents plays them in breaks and they will go live again tomorrow (3/21 10AM PT). But I'm pretty sure it'll break, since the correct #EXT-X-PROGRAM-DATE-TIME's are missing and the player can't play the (not existing) .ts segment.

[PhiZero]

@pixeltris

I tried your solution with a stream that does frequent mid-rolls, and it broke the feed. After switching tabs it restarted again when I switched back, but I think that's unrelated. It works fine for pre-rolls however.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/ddffe9cfabcf881a028a4a813817ade96f6bad17/twitch_hls_test.js this version should display a message in the top left corner "uBlock Origin is waiting for ads to finish..." and the stream should also now continue after midroll ads finish (though the video feed will freeze until the next stream segment is available, I could possibly modify it to go to black or something?).

I might strip out some of the changes I made to support displaying custom .ts files as it really isn't necessary. But besides from that I think it should be good enough?

@stoically Twitch seems to be very finicky about what it can load, I couldn't get anything from ffmpeg or vlc's convert tool to work.

EDIT: https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/58cf8f1e503f3b5323daf6c12caf986decc86e5e/twitch_hls_test.js this one should be a tiny bit smoother when midrolls play. Live segments take a while to show up during midrolls which is unfortunate. Also I think there is currently some issue when midrolls finish where it skips a couple of segments (possibly due to the sequence modifications). I have noticed that during some hours of the day simply refreshing can result in no ads so it might be possible to spam the access_token request until the result is ad-free (this would require a poll to access_token, followed by the main m3u8, then finally the stream m3u8 to see if ads there or not).

[stoically]

@pixeltris Works fine for Pre-roll ads - have yet to test with Mid-rolls, I'll let you know. Though it might lead to unexpected behavior when switching streams, since self.seq and the other self assigned variables keep their values (related: https://github.com/instance01/Twitch-HLS-AdBlock/issues/12).

To generate playable placeholder .ts segments, it seems necessary to include a valid "timed metadata" (timed_id3) stream. Here's some Pre-roll debug output from ffprobe for 8 initial ad segments and 2 following live segments - including their corresponding m3u8 entries and the raw timed_id3 header.

[pixeltris]

I've found streams under the "Always On" directory are a good resource for midroll ads (they show ads every 10 minutes or so). Is there a suitable place to reset self values?

[stoically]

It seems /access_token is requested for every new stream, so checking for that in the fetch replacement and resetting the self values there might do the trick.

[stoically]

Btw, textStr.includes(',live\n') will probably not work for VODs or Re-Runs - could be replaced by .includes('#EXT-X-SCTE35-IN').

[pixeltris]

I used ,live\n because when midrolls are played there are live segments followed by ad segments. Maybe a better check would be if either #EXT-X-SCTE35-IN exists or #EXT-X-SCTE35-OUT comes after a #EXTINF tag. /access_token is a fetch that occurs outside of the worker context so I guess window.fetch would need to be redirected and then a message passed to the worker?

[stoically]

Sounds good.

Yeah, I guess it needs overwriting of fetch in the main script, checking for /access_token, sending a navigator.serviceWorker.controller.postMessage and having a self.addEventListener('message', listener) in the serviceworker.

[pixeltris]

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/029a04d6bd8e49c5d37bb086c5b00d4125a70305/twitch_hls_test.js added the suggested changes. I'm still not sure what causes some live segments being skipped when a midroll finishes.

[TheEnhas]

One thing that someone has done on reddit is simply find the source URL of the ad while it's playing and then add it to the filters, (supposedly) preventing that particular ad from playing again.

Would it be possible for uBO to dynamically create a Twitch ad blacklist on the fly as ads are triggered? You'd still need to block them via the scriptlet the first time you get that particular ad, but after that you shouldn't get the same ad again.

[pijcab]

@pixeltris im still getting pre-rolls 😞

[pixeltris]

@pijcab it should pretty aggressively remove all ads. I assume either the custom resource didn't load at all or there is exception in the code somewhere which causes the script to break down. I have only tested on chrome so far.

I have been playing around with the idea previously mentioned of requesting the m3u8 until no ad is shown. If you keep requesting the main m3u8 (the one with the stream quality m3u8 lists) you will eventually get a response without any ads (depending on your location and how aggressively Twitch is serving ads. from my rough testing it only takes a few requests). Using this method the stream loads much faster than stripping the ad segments (pretty much instantly if you thread the requests). I'm not too sure how to implement in javascript though, I have been testing it via a proxy server.

[pijcab]

@pixeltris idk if this is the correct way to put multiple user resources, the first one is requiered NanoAdblocker (I separated both with a coma) :

userResourcesLocation https://gitcdn.xyz/repo/NanoAdblocker/NanoFilters/master/NanoFilters/NanoResources.txt, https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/029a04d6bd8e49c5d37bb086c5b00d4125a70305/twitch_hls_test.js

This might be the reason I'm still getting them and also it seems to happen much more often on the Twitch welcome page

[stoically]

@pixeltris One possible way to do it might be something along the lines of

function fetchTs() {
        var realFetch = fetch;
        fetch = async function(input, init) {
            if (typeof input === 'string' && input.includes('/api/channel/hls/')) {
                while (true) { // a for loop with a maximum of retries might be the better choice
                    var encodingsM3u8Response = await realFetch.apply(this, arguments);
                    var encodingsM3u8 = await encodingsM3u8Response.text();
                    var streamM3u8Url = encodingsM3u8.match(/^https:.*\.m3u8$/m)[0];
                    var streamM3u8 = await realFetch(streamM3u8Url);
                    if (!(await streamM3u8.text()).includes('#EXT-X-SCTE35-OUT')) {
                        return encodingsM3u8Response;
                    }
                }
            }
            return await realFetch.apply(this, arguments);
        }
    }

@pijcab userResourcesLocation expects one valid url, so that won't work.

[pixeltris]

Awesome thanks, that seems to work great. I had to make a slight modification as .text(); was consuming the response buffer which would result in an error HTTP Response Error: TypeError TypeError: ReadableStreamReader constructor can only accept readable streams that are not yet locked to a reader under chrome. I haven't tested firefox.

This is an updated version using @stoically's code, this will attempt to get an ad-free stream 3 times. If it fails it will fall back to removing the ad segments. I also changed the code from using body / ReadableStream hooks to just modifying the fetch response as the previous method was causing noticeable stuttering on some streams.

https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/a0c0d84d78d3a087835143f019cb108a2d729ea5/twitch_hls_test.js

The multi attempt m3u8 thing should probably also be extended to midrolls as it is also possible to skip midrolls using that method. I think this would require a full stream reset though as the main m3u8 is only requested when the stream first loads.

[TheEnhas]

Is three tries enough though? Because in my experience I have to reload a stream way more times than that to not get an ad.

As for midrolls, I wouldn't mind it resetting the stream but that would obviously stop the video for a few seconds (which is still better than seeing the ads).

[pixeltris]

I put it at a low number because the requests have to wait for each other which can be slow at larger numbers. If the code were modified to make many requests at the same time it could simply pick the one without the ad.

Try increasing the number as-is and see how it works out for you? Currently I'm rarely getting ads anyway so it's hard for me to tell.

[stoically]

@pixeltris Glad it works and good job with the improvements!

Here's a concurrent version for the skip ad attempts (untested)

```js return new Promise(function(resolve, reject) { var skipAdTries = 0; var skipAdMaxRetries = 5; for (var i = 0; i < skipAdMaxRetries; i++) { (new Promise(async (skipAdResolve, skipAdReject) => { var encodingsM3u8Response = await realFetch(url, options); var encodingsM3u8 = await encodingsM3u8Response.text(); var streamM3u8Url = encodingsM3u8.match(/^https:.*\.m3u8$/m)[0]; var streamM3u8Response = await realFetch(streamM3u8Url); var streamM3u8 = await streamM3u8.text(); if (++skipAdTries === skipAdMaxRetries) { resolve(new Response(encodingsM3u8)); skipAdReject(`max skip ad attempts reached (attempt #${skipAdTries})`); return; } if (!streamM3u8.includes('#EXT-X-SCTE35-OUT')) { resolve(new Response(encodingsM3u8)); skipAdResolve(); } else { skipAdReject(`skip ad (attempt #${skipAdTries})`); } }).catch(console.log); } }); ```

Doing the skip ad attempt for Mid-rolls is a great idea. If it should avoid resetting the stream, we would probably need to store which encoding is currently in use, so it's possible to serve the correct m3u8.

[pijcab]

Ok now that I put only one input into userResourcesLocation your snippets seem to be working. Thing is and, I'm sorry but I don't really know how to verify this, but the uBlock is waiting for ads to finish... shows up and blacks the screen for nothing (when unset in the advanced tab no ad is playing).

Can you guys give me a way to verify that and ad is actually playing ? @pixeltris @stoically

[pixeltris]

Yea it should be a black screen for preroll ads and a frozen screen for midroll ads. When the ads is finished the message in the top left should go away and the stream should start. If no ad is being delivered by Twitch then the stream should just start and there wont be a message at the top left.

I don't think Twitch always delivers ads which is why you might not be getting ads when the value is unset. But again to clarify; if the message in the top left is being displayed you know for certain an ad is being delivered by Twitch (and the multi m3u8 attempt thing failed, but ads are still being blocked).

Here is another update to the code using async requests (it makes one initial m3u8 sync request, then 5 async). Also fixed an issue with seq causing lag. I'm currently not getting many ads so haven't been able to test properly. https://gist.githubusercontent.com/pixeltris/e2d4194d7fd5bfa1df39470cb589a012/raw/7168b80b0ac2ee5ce8ef1ee5cc65f1975081b288/twitch_hls_test.js

[pijcab]

@pixeltris thank you for all the work you put into this, really appreciate it.

However I'll stick to using a VPN for now since it completely disables the SureStream ad injects, without having to wait on a black screen from time to time.

On a side note, the ad distribution seems to be inconsistent (or maybe on time watched ?) making this even harder to develop. Or maybe I don't get it...

[gorhill]

I received this email last night at 11:08 PM GMT-4 -- I don't know why this was sent to me directly rather than posted here, so I will reproduce the email here:

Raymond -

Good to meet you. I am a Principal Engineer at Twitch that focuses on the Video Platform and write to alert and seek your help over a development that could disrupt the Twitch service, thus impacting Twitch users.

I recently came across the following discussion about SureStream and uBlockOrigin that I wanted to reach out to you about:

https://github.com/uBlockOrigin/uAssets/issues/5184

I have concerns around the direction the conversation is taking, and wanted to bring them to your attention as the maintainer of the project.

Reviewing the thread it appears that a set of contributors intend to make multiple master manifest requests on stream load. My concern here is that this puts significant additional load on our services that are invoked at the start of a video session. When large channels come online, or large channels quickly alternate between being up and down (i.e., RTMP disconnect, reconnect scenarios), the extra volume of requests this design imposes will significantly slow down Twitch’s ability to get users back to a live video experience. Essentially, the extra requests would result in a DDoS on our service.

My primary concern is that if the proposed change is merged and goes live it could cause service degradation that will require intervention and request blocking. I am asking for your help to avoid this situation and to steer efforts away from those that could impact our service stability.

I appreciate your attention and feedback, and remain available to discuss. I am going to be flying for the next several hours, but will check my email when I land, and will check several times over the weekend in case you have additional questions.

Cheers, [Name redacted]

By the way, I unsubscribed from this thread a few days ago, as I was constantly notified about it. I figured someone would notify me by name once a solution is identified, then I would evaluate whether I agree with the solution.

Now I want to make this clear: my position has absolutely nothing to do with the email from the Twitch engineer. When a solution for one specific case crosses a threshold of complexity, I prefer to refuse it and instead encourage such solution to be integrated into its own specialized extension, this is true for other cases too, for example the famous Facebook ads using invisible text to fool blockers. I consider the solution in the gist above to cross that complexity threshold.

[bogachenko]

Essentially, the extra requests would result in a DDoS on our service.

Tell me how YOUR ads to block and DDoS will not be!

[stoically]

the extra volume of requests this design imposes will significantly slow down Twitch’s ability to get users back to a live video experience

I wonder if it'll take longer than waiting for ads to finish.

@gorhill Thanks for bringing that to our attention. I have a question regarding the complexity threshold: would you consider overwriting the ServiceWorker or injecting logic which does the additional "master manifest requests" (using fetch) as already crossing the threshold or is it mostly the LOC? I'm asking because the proposed solution could be significantly trimmed down in terms of LOC, leaving out some edge-cases that would no longer be handled.