Open Mayo77 opened 4 years ago
Got the same issue, don't know how to fix :(
Did you ever figure out how to fix this?
i'm using https://github.com/grrttedwards/wow-addon-updater now, you do need to install more stuff to make it work tho
Hi, The issue is, that curse changed to cloudflare protection. There is a cf-1020 error response. For Cloudflare the python package cscrape exists. Change all request.get calls to cscraper calls. The HTML page changed, therefore the "Searches needed to be adopted".
Following changes need to be applied to SiteHandler.py and worked for me the last 3 days. No real blocking issues.
import cfscrape scrapper = cfscrape.create_scraper(delay=10) # returns a CloudflareScraper instance
def curse(addonpage): if '/datastore' in addonpage: return curseDatastore(addonpage) try: page = scrapper.get(addonpage + '/download') page.raise_for_status() # Raise an exception for HTTP errors contentString = str(page.content)
#endQuote = contentString.find('"', indexOfZiploc) # Will be the index of the ending quote after the url
# New Cloudflare 28.Sep.2019
indexOfZiploc = contentString.find('automatically, click ') + 30 # Will be the index of the first char of the url
endQuote = contentString.find('"', indexOfZiploc) # Will be the index of the ending quote after the url
return 'https://www.curseforge.com' + contentString[indexOfZiploc:endQuote]
except Exception:
print('Failed to find downloadable zip file for addon. Skipping...\n')
return ''
def getCurseVersion(addonpage): try: page = scrapper.get(addonpage + '/files') page.raise_for_status() # Raise an exception for HTTP errors contentString = str(page.content)
#endTag = contentString.find('</span>', indexOfVer) # ending tag after the version string
# New Cloudflare 28.Sep.2019
indexOfVer = contentString.find('text-primary-500 text-lg') + 26 # first char of the version string
endTag = contentString.find('</h3>', indexOfVer) # ending tag after the version string
##################
@wsch-wa thank you for this. I'll note that getAddon() also needs updating in WoWAddonUpdater.py (import cfscape, create the scraper object then replace requests.get with scraper.get).
I'd tried manually reproducing the exact request headers and user agent from Firefox and somehow it still gave the 403. No idea how Cloudfare does it (Javascript in FF was disabled for curseforge, my Python request really seemed identical) but I'm not a web dev so must be missing something.
Your fix works, cheers.
Hi, I have uploaded my changes for further reference. Hope that Helps.
Greetings
I had this error since 8.2.5 went live anyone know how to fix it?
Error:
And this happens to ALL my addons exact same error every single time.
Anyone know how to fix this please?