lossless1024 / StreaMonitor

Adult live stream downloader for advanced people. I could have chosen a better name.
GNU General Public License v3.0
172 stars 42 forks source link

Chaturbate: Rate limited #147

Open Maxbbc opened 1 month ago

Maxbbc commented 1 month ago

The manager seems to check the ONLINE status for each model in the list which may result in multiple requests to the site in a short amount of time, possibly flagged as spam activity. Docker container shows in the log messages looking like: yyyy-mm-day HH:mm:ss- INFO - [CB] model: Rate limited.

This can be avoided by requesting the list of models ONLINE, and then check whether models of interest are in the lines of connected models. Add delay between successive requests, such as o reduce the total number of requests per min.

Ro-Den commented 1 month ago

The manager seems to check the ONLINE status for each model in the list which may result in multiple requests to the site in a short amount of time, possibly flagged as spam activity.

You'll get Cloudflare challenge for this, not rate limiting, I guess.

Maxbbc commented 1 month ago

It is not clear to me what rate limited even means.

DerBunteBall commented 1 month ago

The error "Rate Limit" means in a narrower sense that the API does not respond to the request. Originally because the API detects too many requests, for example.

However, the error also occurs when something unexpected happens. The source code does not differentiate properly here.

It is not entirely unlikely that @Ro-Den suspects the right thing.

CB turned up the Cloudflare filtering not too long ago - at least it felt like it.

Test this (change "somemodel" to a valid room_slug):

curl -X "POST" "https://chaturbate.com/get_edge_hls_url_ajax/" \
     -H 'X-Requested-With: XMLHttpRequest' \
     -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0' \
     -H 'Content-Type: application/x-www-form-urlencoded; charset=utf-8' \
     --data-urlencode "room_slug=somemodel" \
     --data-urlencode "bandwidth=high"

If the previous command returns HTML and no JSON, the error has been found. Then the IP runs into the filtering (JavaScript or Captcha Challenge). This usually happens with data center, hoster, VPN and proxy IPs. In addition, IPs from unclean networks enter the filter (spam, DDoS and other conspicuous network activities).

You need a clean home ISP IP to stand any serious chance.

Ro-Den commented 1 month ago

Thanks to Cloudflare, it's close to impossible nowadays to find a free working proxy for CB. And the working ones won't last long (a couple of days or weeks at best).

Cloudflare's challenge is not so easy to solve, so there's no public information on how to automate the process.

Taking into account geo-blocking the days of easy solutions (like this one) are gone. Depending on the region even with "a clean home ISP IP" you may loose up to 40-50% of "content".