github 403's with reconcile --installed

[ciphersimian]

This may be more of a question / feature request, but I suspect this is going to be a common issue for anyone coming back for patch 9.2.5 after the CurseForge shutdown of the Twitch API.

When I did the reconcile --installed mentioned in the Pinned issue it found pretty much all the addons I care about on either wowi or github, but then when it went to install at the end the majority of the github ones got a 403, probably just from hitting github too hard and all at once:

! github:stanzilla/advancedinterfaceoptions
  internal error: "403, message='rate limit exceeded', url=URL('https:
    //api.github.com/repos/stanzilla/advancedinterfaceoptions')"
! github:vladinator89/wow-addon-candybuckets
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/vladinator89/wow-addon-
    candybuckets')"
! github:dragnogd/instance-achievement-tracker
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/dragnogd/instance-
    achievement-tracker')"
! github:sfx-wow/masque
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/sfx-wow/masque')"
! github:sfx-wow/masque_caith
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/sfx-wow/masque_caith')"
! github:caedilla/rsa
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/caedilla/rsa')"
! github:weakauras/weakauras2
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/weakauras/weakauras2')"
! github:bigwigsmods/ora3
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/bigwigsmods/ora3')"
! github:astralguild/astralkeys
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/astralguild/astralkeys')"
! github:wowrarity/rarity
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/wowrarity/rarity')"
! github:nevcairiel/routes
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/nevcairiel/routes')"
! github:exochron/toyboxenhanced
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/exochron/toyboxenhanced')"
! github:nevcairiel/handynotes
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/nevcairiel/handynotes')"
! github:urtgard/wqachievements
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/urtgard/wqachievements')"
! github:tercioo/plater-nameplates
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/tercioo/plater-nameplates')"
! github:raiderio/raiderio-addon
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/raiderio/raiderio-addon')"
! github:vendethiel/gladiusex
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/vendethiel/gladiusex')"
! github:casualshammy/nameplatecooldowns
  internal error: "403, message='rate limit exceeded', url=URL('https:
    //api.github.com/repos/casualshammy/nameplatecooldowns')"
! github:deadlybossmods/dbm-pvp
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/deadlybossmods/dbm-pvp')"
! github:deadlybossmods/dbm-legion
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/deadlybossmods/dbm-legion')"
! github:funkydude/basicminimap
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/funkydude/basicminimap')"
! github:funkydude/bugsack
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/funkydude/bugsack')"
! github:kiatra/blizzmove
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/kiatra/blizzmove')"
! github:p3lim-wow/interactivewormholes
  internal error: "403, message='rate limit exceeded',
    url=URL('https://api.github.com/repos/p3lim-
    wow/interactivewormholes')"

Perhaps this can be throttled to prevent this issue? Now I have the problem that it uninstalled all the curse versions, but failed to install the ones above, so I think I'm stuck manually reinstalling each one now?

[ciphersimian]

Doing a bit more research on this it seems GitHub has a rather low unauthenticated user rate limit of 60 requests/hour/ip. This question and answer gives various options for authentication: https://stackoverflow.com/questions/33655700/github-api-fetch-issues-with-exceeds-rate-limit-prematurely so I guess what is needed here is a way to pass a GitHub auth token to instawow to get the 5000 requests/hour/ip rate limit that authenticated users have.

It would also probably be best if instawow doesn't remove a reconciled addon until the replacement has been successfully installed.

[ciphersimian]

OK, poking around I found that instawow already has a github auth feature, I just had to run through configure again.

With a bit of regex foo I was able to turn the above into a list to feed back into install so I think I'm good now. Perhaps you can still find something worth improving here so I'll leave this open, but if you don't see the value in it feel free and close. Thanks and sorry for all the noise as I worked my way around to a solution.

[layday]

You can generate a GitHub access token by running instawow configure access_tokens.github without having to re-configure anything else. This would be worth mentioning in the readme (in addition to the configure command's help text). I agree that this behaviour is undesirable, but it only becomes an issue during reconciliation. There are some workarounds we might wanna consider, like fetching new add-ons before uninstalling the old ones.

[layday]

Fixed in v4.3.0.

That only took two years.