New parser

[orels1]

New parser concept

• User logs in through GH (granting public repo and webhook creation access)
• User chooses a repo from the list (only public ones are supported) to be added as an unapproved
• Red-Portal creates a webhook on cogs.red/api/wh/github
• Red-Portal recieves data from GH on repo change and looks if any of the info.jsons were affected
  • If so - Red-Portal triggers parser to update the repo

Admins and staff will still be able to trigger the parser manually (the system-wide repo-parse is only available to admins), but the main way to update will be to trigger on significant repo updates

Goals achieved:

• Less GH api calls
• Responsiveness of Red-Portal
• This approach allows cog creators to add their repos themselves

Cons:

• Repos could not be manually added by staff anymore (needs discussion)

Internal structure

• [ ] GH oauth flow (won't use Passport this time - no need)
• [ ] JWT auth for the website (may use the same "insta-death" cookie approach as the last time)
• [ ] User profile section with added repos list
• [ ] Webhook integration (with proper update checks, so we won't trigger when we don't need to)
• [ ] Use of personal tokens to gather data for user's repos (virtually eliminates a chance to ratelimit Red-Portal)
• [ ] Admin panel with ability to parse, hide, or delete repos and their cogs
• [ ] README.MD support for repos and cogs

[palmtree5]

"Significant repo updates":

• [ ] New cog
• [ ] Renamed cog
• [ ] Deleted cog
• [ ] Any change to any file named info.json in the repo

Do I have this right or am I missing something?

[orels1]

Yes, although if the New Cog doesn't have an info.json I won't add it.

[Kowlin]

Shit I did it again! I looked at the issue and replied 2 days later! It looks good to me, but we can probably resolve the last con, I actually don't think its a con, the only reason we add those repo's is on request. However you do open a path to a giant spam of repo's build as Red extensions or not, so I do think that we need some basic verification steps. Let a staff member review the repo? How will we keep up to date about this? Webhook ;)

[orels1]

Well, I mean, there is a basic review process - if the repo contains info.json, at least one cog and that cog also has an info.json, I think that's enough, 'cause currently that's what I check before adding to the "unapproved" category

About keeping up-to-date, I will create a channel on my own server with something like "cogs.red changes" to spam events like "repo added" and "repo type changed".

This seems to be good enough

[Kowlin]

Sounds fair, now if I can get a invite to this mythical server, we will be set ;)