kuhnertdm / wow-addon-updater

Python script for mass-updating World of Warcraft addons
GNU General Public License v3.0
117 stars 28 forks source link

Failed to find downloadable zip file #54

Open Mayo77 opened 4 years ago

Mayo77 commented 4 years ago

I had this error since 8.2.5 went live anyone know how to fix it?

Error:

Failed to find downloadable zip file for addon. Skipping...

Failed to find version numbers for : https://www.curseforge.com/wow/addons/askmrrobot Installing/updating addon: askmrrobot to version: Failed to find downloadable zip file for addon. Skipping...

And this happens to ALL my addons exact same error every single time.

Anyone know how to fix this please?

duivel111 commented 4 years ago

Got the same issue, don't know how to fix :(

Mayo77 commented 4 years ago

Did you ever figure out how to fix this?

duivel111 commented 4 years ago

i'm using https://github.com/grrttedwards/wow-addon-updater now, you do need to install more stuff to make it work tho

wsch-wa commented 4 years ago

Hi, The issue is, that curse changed to cloudflare protection. There is a cf-1020 error response. For Cloudflare the python package cscrape exists. Change all request.get calls to cscraper calls. The HTML page changed, therefore the "Searches needed to be adopted".

Following changes need to be applied to SiteHandler.py and worked for me the last 3 days. No real blocking issues.

import cfscrape scrapper = cfscrape.create_scraper(delay=10) # returns a CloudflareScraper instance

def curse(addonpage): if '/datastore' in addonpage: return curseDatastore(addonpage) try: page = scrapper.get(addonpage + '/download') page.raise_for_status() # Raise an exception for HTTP errors contentString = str(page.content)

indexOfZiploc = contentString.find('download__link') + 22 # Will be the index of the first char of the url

    #endQuote = contentString.find('"', indexOfZiploc)  # Will be the index of the ending quote after the url
    # New Cloudflare 28.Sep.2019
    indexOfZiploc = contentString.find('automatically, click ') + 30  # Will be the index of the first char of the url
    endQuote = contentString.find('"', indexOfZiploc)  # Will be the index of the ending quote after the url
    return 'https://www.curseforge.com' + contentString[indexOfZiploc:endQuote]
except Exception:
    print('Failed to find downloadable zip file for addon. Skipping...\n')
    return ''

def getCurseVersion(addonpage): try: page = scrapper.get(addonpage + '/files') page.raise_for_status() # Raise an exception for HTTP errors contentString = str(page.content)

indexOfVer = contentString.find('file__name full') + 17 # first char of the version string

    #endTag = contentString.find('</span>', indexOfVer)  # ending tag after the version string
    # New Cloudflare 28.Sep.2019
    indexOfVer = contentString.find('text-primary-500 text-lg') + 26  # first char of the version string
    endTag = contentString.find('</h3>', indexOfVer)  # ending tag after the version string

##################

Ulturine commented 4 years ago

@wsch-wa thank you for this. I'll note that getAddon() also needs updating in WoWAddonUpdater.py (import cfscape, create the scraper object then replace requests.get with scraper.get).

I'd tried manually reproducing the exact request headers and user agent from Firefox and somehow it still gave the 403. No idea how Cloudfare does it (Javascript in FF was disabled for curseforge, my Python request really seemed identical) but I'm not a web dev so must be missing something.

Your fix works, cheers.

wsch-wa commented 4 years ago

Hi, I have uploaded my changes for further reference. Hope that Helps.

Greetings

Cloudflare-Changes-Files.zip