jtkencode / cp-leaderboard

0 stars 0 forks source link

Calculate RCPC token on scraping Codeforces pages #14

Open saifulwebid opened 3 years ago

saifulwebid commented 3 years ago

Since last night, the Codeforces scraper has been failing with this AssertionError:

Traceback (most recent call last):
  File "./bin/scrape-codeforces", line 47, in <module>
    assert len(leaderboard_rows) >= 6 + 1
AssertionError

On printing the soup variable, we got this:

<html><body>Redirecting... Please, wait.<script src="/aes.min.js" type="text/javascript"></script><script>function toNumbers(d){var e=[];d.replace(/(..)/g,function(d){e.push(parseInt(d,16))});return e}function toHex(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}var a=toNumbers("e9ee4b03c1d0822987185d27bca23378"),b=toNumbers("188fafdbe0f87ef0fc2810d5b3e34705"),c=toNumbers("bd4a322f6a6a0838c67ed4662a4a2fce");document.cookie="RCPC="+toHex(slowAES.decrypt(c,2,a,b))+"; expires=Thu, 31-Dec-37 23:55:55 GMT; path=/";document.location.href="https://codeforces.com/ratings/organization/3732?User-Agent=cp-leaderboard+%28https%3A%2F%2Fgithub.com%2Fjtkencode%2Fcp-leaderboard%29&f0a28=1";</script></body></html>

It seems that now Codeforces redeploy their basic security measures as described in these posts:

We need to calculate the token based on the first response from Codeforces and use that token to scrape the ranking page.

saifulwebid commented 3 years ago

I've disabled the scraper temporarily on #15.