Open saifulwebid opened 3 years ago
Since last night, the Codeforces scraper has been failing with this AssertionError:
AssertionError
Traceback (most recent call last): File "./bin/scrape-codeforces", line 47, in <module> assert len(leaderboard_rows) >= 6 + 1 AssertionError
On printing the soup variable, we got this:
soup
<html><body>Redirecting... Please, wait.<script src="/aes.min.js" type="text/javascript"></script><script>function toNumbers(d){var e=[];d.replace(/(..)/g,function(d){e.push(parseInt(d,16))});return e}function toHex(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}var a=toNumbers("e9ee4b03c1d0822987185d27bca23378"),b=toNumbers("188fafdbe0f87ef0fc2810d5b3e34705"),c=toNumbers("bd4a322f6a6a0838c67ed4662a4a2fce");document.cookie="RCPC="+toHex(slowAES.decrypt(c,2,a,b))+"; expires=Thu, 31-Dec-37 23:55:55 GMT; path=/";document.location.href="https://codeforces.com/ratings/organization/3732?User-Agent=cp-leaderboard+%28https%3A%2F%2Fgithub.com%2Fjtkencode%2Fcp-leaderboard%29&f0a28=1";</script></body></html>
It seems that now Codeforces redeploy their basic security measures as described in these posts:
We need to calculate the token based on the first response from Codeforces and use that token to scrape the ranking page.
I've disabled the scraper temporarily on #15.
Since last night, the Codeforces scraper has been failing with this
AssertionError
:On printing the
soup
variable, we got this:It seems that now Codeforces redeploy their basic security measures as described in these posts:
We need to calculate the token based on the first response from Codeforces and use that token to scrape the ranking page.