Capture instances become blocked by CB

[insanity54]

One of my servers is showing 403 error when running ytdl.

I'm assuming the repeated ytdl retries over a long period of time end up triggering CB defenses.

3 is a potential fix. Instead of retrying after a delay when ytdl fails, we can listen in on CB realtime events and start when we read a stream start event. This would cut down on the requests to CB and potentially not become blocked.

[insanity54]

Apparently it's not just one server that got blocked by CB. It's all well known cloud services, including VPNs. Big oof, missed two streams in a row because all my servers are blocked. I can't record from home-- don't have the internet bandwidth. Going to have to find a new way of capturing VODs.

[Linden10]

Wow that's a shame that CB is blocking your servers including vpns you use! So every time you try to use yt-dlp you get an 403? I see...I recall that happening to me when I tried yt-dlp as well...

However that changed when I tried CTBrecorder, have you tried that program? I'm not sure how you can get it to run on a ser...OH WAIT they have a server version of the program so it's possible!

They just released version 5.0.1 a couple days ago too! The program comes with a documented help section on how to use, with videos on setting up the servers too! (You can open it in the help section or settings section of the program using the gui)

The only question is if you're still able to access CB...even though yt-dlp gets blocked...if you can access the website, that program should work for ya!

Otherwise if you're still blocked then...I wonder if using a smartdns will work? I'm not sure.

[insanity54]

I think CB added some cloudflare protections in front of their servers. yt-dlp, CTBrecorder, yt-dl all work from my home internet, but not from my VPS.

It's not the programs being blocked, it's the IP ranges of my requests being blocked. All requests from my VPS are responded to with a Cloudflare challenge webpage meant to allow only humans to proceed.

I looked at ctbrec's sourcecode to see how it worked. It's using ffmpeg under the hood, which is the same thing ytdl and yt-dlp are using. ctbrec does UA spoofing so that might help in some cases, but not against the cloudflare protections that require running javascript and solving a CAPTCHA.

I saw the docs for CTBrecorder's server, but the executable mentioned in those docs is missing. IDK if it was removed in a later release and the docs haven't been updated, but there isn't a server that I can see. Maybe it's activated using a CLI flag now? IDK.

I've seen this sort of thing before with other websites when they don't want people visiting via VPN. They just block all IP addresses of all known VPN endpoints. That's 99% of the time an IP belonging to a cloud provider. Same thing goes for blocking bot traffic. Meanwhile residential internet users are allowed access.

Solutions I see going forward are to find a small VPS provider who isn't well known. There's a possibility that the IP addresses they have are not on cloudflare's blacklist. This is not something I think is feasible because it means renting services and testing if it isn't blocked. That can get really expensive really fast. And it also means that the requests from that VPS have to be limited otherwise that IP can get blocked.

Other solution is to rely on viewers to submit vods. That's pretty much my only option at the moment since it's impossible to watch in 1080p on my home internet. Too much buffering. ctbrec has a feature for restricting the recording resolution, but I think CB's servers override that and it immediately gets upgraded to the highest resolution.

[Linden10]

Hmmm so it's cloudflare that's blocking you...I recall something that could probably mitigate that... Maybe this will work?

CloudProxy or cloudscraper

If not, you should do some research into how to bypass it, like say using a smartDNS or cloudflare's WARP program or a vpn that's not blocked. It be better for ya then to waste a lot of money on another vps that could end up not working again in the future.

Also you couldn't find the server executables? Have you downloaded the server-unified program? It's not part of the CTBrec linux/win/mac but it's own archive "server-xxx-unified.zip" in which you'll find the executables server.bat (windows), server-linux.sh, and server-mac.sh!

You can download it in the link above! Hope this helps ya out!

[Linden10]

Ok, I think I found a way to get the vods that you weren't able to record the past two streams... so when I get them, I'll help ya out and provide them!

I'm using Recurbate and a userscript I found to do it, it'll take time since it requries waiting 1 day per video but when I'm done I'll upload them to mega for ya!

[insanity54]

That's very kind of you, thank you! I'm looking into the solutions you suggested.

[Linden10]

After some trickery and trial and error, I was able to get the vods much quicker then usual from recurbate! Here you go, the vods!

March 23, 2023 Stream: 1: https://mega.nz/file/NAg3VAjD#iZqIEFrzB6Tu_cuOffEYTwh2pwR4B2TZfZwRjO1iGvM 2: https://mega.nz/file/gM5yUCKZ#4vevt3I5AxEWc3u9O87WLPzNNxklDR9qP3G0UW9-rUg

March 24, 2023 Stream: https://mega.nz/file/cdwyjSpb#eH3Nyi-6mce1dF_jxPrPElR5BNGwmh6t3EBC2GWdYYE

They're both 1080p 60fps which is great! Your welcome!

[insanity54]

Got a lead on a potential bypass from a tech demo talking about browsing the internet via Tor while avoiding CAPTCHAs.

https://privacypass.github.io/

[insanity54]

worked around the issue for the time being. closing for now