Github Integration no longer updating issues on installed repos

[PromoFaux]

Admin edit: $100 bounty (or more eventually) on helping get this issue fixed - see https://www.bountysource.com/issues/43409489-github-integration-no-longer-updating-issues-on-installed-repos - yes having to edit this in manually rather than rely on the GitHub integration itself may count as irony? Hard to be sure!

Hi there,

We had this all working for some time about a year ago on https://github.com/pi-hole/pi-hole and https://github.com/pi-hole/adminLTE, but around 10 months ago (I think) it stopped working.

I've just installed the integration on a personal repo, and it appears to work there, so not really sure what has happened with the existing repos.

Is there a way to uninstall integration?

[Yehonal]

my repo too is not working anymore, we've multiple bounties but no integration with git, it's very painful

[Yehonal]

P.S. Why bountysource is 90% bugged lately? people is putting money not candies on issues. Please

[VanillaNinjaD]

I recently had a developer recommend bountysource as a motivational tool. Shortly after that, I pledged to an open GitHub project issue and my money disappeared, I don't see it under my pledge history and it is not attached to the GitHub issue. I have sent an email to support@bountysource.com but have heard nothing back. Needless to say its frustrating at the least.

[marvin-bitterlich]

@rappo any news on this issue? We got hit by it too (it worked for some months). Would posting a bunty on this help?

[snipe]

We're having this issue too. Have tried disabling the integration in Tools and then re-enabling, but the bountysource footer never shows up on issues, and no integration with githib seems to be working. I'd be happy to try reinstalling it in case it would help, but I don't see a way to delete the repo to re-add. It's been broken for some time.

If there's anything I can do to help with this issue, I'd be glad to. The Bountysource integration wasn't used by everyone, but was appreciated by those that did use it.

/me waves at @VanillaNinjaD

[PromoFaux]

@rappo an official response on this would be great, thanks. The integration used to be excellent, and since it stopped working, we have missed a couple of bounties that were added to open issues. Those issues have since been closed with no resolution, which seems unfair to those that backed the bounties in the first place.

What can I do, or ask you or the team to do, in order to get this working again?

[Cervator]

Alright I'm not quite as official as @rappo but until he digs out more from his day job maybe we can all try to triage this one together :-)

So the issue at hand seems to be that existing repositories where you went to the "Tools" menu in the past (maybe even the long past like nearly a year ago) and hooked up the integration that decorates an issue no longer works. New repositories that have never before been hooked up work fine if you install the integration.

A related issue is #1062 which documented a GitHub change in parsing stuff put into issues that made the integration post some now-broken markup. So it was still working, but what it was posting wasn't quite right anymore. Coincidentally (maybe) this was noted about 9 months ago.

In February, about 5.5 months ago, a fix in #1068 was submitted and merged, then deployed probably shortly after and the inserted footers started behaving again.

Not long after / around the same time some of the affected users started noting that the integration wasn't posting footers at all any longer to repos previously integrated. So it seems to have been broken in February/March, at least for some users. Maybe that deploy could be involved. It would be interesting if anybody could confirm having footers not getting posted in existing repos prior to that date. It sounds like it was still working in some cases, although maybe confusion over the incorrect (but working) footers is involved.

For some experimenting from me: I had lots of old repos enabled in the past and we got hit by the broken footer issue, so in trying to fix it we edited some, enabled/disabled the tool per repo, etc. I just tried a brand new never previously touched repo and the integration worked perfectly fine.

I then tried to revoke the Bountysource OAuth app from using my account (personal settings -> Authorized OAuth apps), confirming that the new repo no longer would get the footer posted in a new issue and I had to log out, log back in, re-authorize the app, and then I could successfully save the integration again. No dice on the old repo, still didn't get footers. No success on the new repo either after I had revoked and re-authorized.

So if a bunch of users after the footer started posting incorrectly attempted the revoke path (or did so for any other reasons, like simply cleaning out old tokens to then only renew what you actively used) then any old repo would likely stop working, permanently, as something doesn't clean up fully in the revoke-reauthorize flow. That would also explain some users reporting no problems (may have re-saved the integration but not revoked/reauthorized) while others could not get the footer to appear in anything but newly integrated repos.

Can anybody poke holes into that theory? Has anybody been able to post a new issue on a repo activated before a revoke, then see a footer get edited in?

If not maybe that's a spot in the code we can search for. Maybe existing integrations never try to renew something after a revoke, so data goes dirty and can't be reset. That sounds like a bug we can find.

Alternatively/additionally the tools page could really use the addition of a button (or two) to outright remove or reset an integration for a given repo. As noted there is no way to completely clean up the integration at present, only re-save the settings. Adding functionality to outright remove it and all related data in the DB probably would also get around this issue. Reset instead of remove could simply remove then re-add in one action. You could then also try to re-add the integration using a different account, like one you're using for bot actions rather than a personal one.

A completely different (but far more complicated) option might be looking into making a proper GitHub app for the integration instead. The Bountysource OAuth app for the integration is 5 years old at this point.

Maybe one of the recent volunteers interested in becoming a core dev could try this bug out. Will Drop a link to it on Slack/IRC as well, maybe channel ping in a few days if it doesn't have any takers yet.

[snipe]

I then tried to revoke the Bountysource OAuth app from using my account (personal settings -> Authorized OAuth apps), confirming that the new repo no longer would get the footer posted in a new issue and I had to log out, log back in, re-authorize the app, and then I could successfully save the integration again. No dice on the old repo, still didn't get footers. No success on the new repo either after I had revoked and re-authorized.

I can confirm this behavior on our (older) repo. Our footer/badge integration stopped working some time ago. I tried unchecking the boxes in the tools section to "disable" the integration via BS. I then waited a few days (in case there was any synchronization that had to happen on the BS side), and re-checked those boxes. Waited another few days, with no change.

I tried revoking via GH, logging out of BS, logging back in, re-adding it, no dice. (I have my attempts documented in another issue, but I can't find it at the moment, and just wanted to confirm that this is what I'm seeing as well. )

I can tell you that I'm the only person who futzes with the repo in Bountysource, so there isn't a possibility of conflicting BS/GH Repo Owner actions.

I do wish I could remember when they stopped working for us, but I'm honestly not sure. I'm sorry.

[PromoFaux]

It would be interesting if anybody could confirm having footers not getting posted in existing repos prior to that date.

So according to a quick search of our open and closed issues, the last time the integration worked (for us, atleast) was october 2016

[Cervator]

Just added a $100 bounty on this, not that you'd be able to rely on the GitHub plugin to let you know since ... yeah this issue is about getting that fixed :D

While reworking the OAuth app / plugin into an outright GitHub Integration would be nicer, that's probably a fair bit more work. Chances are something just changed with the GitHub OAuth stuff over the years and the token revocation thing wasn't considered (or done right) when the integration was originally written.

[BinaryFissionGames]

So I've been looking into this issue, and here's how I understand it. The flow is something like

Revoke authorization -> BountySource tries to update the repo, which then locks the plugin (this makes it silently fail whenever it tries to update again) -> reauthorization happens, but doesn't unlock the plugin.

So, I've got a few ideas on how to solve it.

1. Allow users to unlock the plugin at their own discretion. 2 functions would be added to the V2, one to view the locks, one to disable the locks. Right now, the locks can only be accessed by admins through the V0 API. When viewing your plugins, you would see some locked tag on locked plugins, and unlock them by pressing a button. This idea is good because there are errors that can occur other than an invalid oauth token error that can cause a lock. I feel there should at least be an indication of this on the user end.

2. Simply clear the locks on all installed plugins when it's reauthorized. This would still have some lock clearing function in the API, but it could be restricted to only invalid oauth token errors.

I like #1, mainly because I don't see why this locking mechanism is hidden behind the admin page.

It shouldn't be necessary to "uninstall" the plugin, I don't think. I could work on adding it though if the previous ideas don't work, or if it's still desired.

This is, assuming there is no natural way of the locks clearing that I didn't see, and that the locking mechanism isn't some dev only build thing.

Thoughts? @Cervator

[Cervator]

Nice catch @Denu8thell ! Where did you find that? I didn't even realize there was a lock system like that. Not that I've dug very deep so far.

Your first suggestion sounds fine, main concern I'd have is whether it would be a hassle to develop and/or if it would be a hassle for users to discover the problem and solve it on their own after the updates are made.

With GitHub integrations released I imagine the long term fix would be a new non-user based application to replace the old plugin. So I wouldn't want anybody to sink too much effort into the old plugin. Although we do need it to work again ASAP :-)

My thought on uninstalling the plugin was in case that helps reset whatever gets stuck. If an enhancement to interact with locks fixes the issue entirely then yeah outright uninstall becomes lower priority.

[BinaryFissionGames]

Alright, I'm gonna work on the first proposed solution then. I really don't think it should be so difficult to implement, and even if it is, I got some time to spend on it.

I don't really know what to do about helping users figure out that this is even a problem, particularly after the updating. I'll think about it.

[Cervator]

At least we can update the documentation and point users to it when the issue occurs. Lots of users probably use it without ever revoking in the first place.

Thanks! :-)

Let me know if you'd like a Slack invite somewhere, just need to get an email address to me or @rappo (maybe PM on IRC)

[BinaryFissionGames]

@Cervator I have a pull request with an initial fix to this problem, if you or anybody else could find time to review it, that'd be great. I'll try to get an email to you on IRC in a bit.

[Cervator]

Yep thanks I saw it - trying to poke @rappo to check it out :-)

[PromoFaux]

Ping! Any update?

[Cervator]

Pong :-)

So the GitHub API issue has been resolved so that was a pretty important step to get out of the way.

Now that shouldn't be an issue in testing out and merging this. Just need @wkonkel or @rappo to take a look, merge, and deploy

[VanillaNinjaD]

It looks like there have been 3 or 4 more issues relating to the Github integration. I spied one pull request related to fixing the issue but not much other movement. Granted I just checked in today when I got the ping

https://github.com/bountysource/core/pull/1130

[VanillaNinjaD]

@cervator you are fast!!!

[snipe]

Awesome, thanks! Any idea when that PR will be able to reviewed/merged?

[Cervator]

Right now most of what little admin effort is available is being consumed by end-of-year stuff - taxes, finance docs, that sort of thing. But I'm trying to highlight the PR associated with this as top of the list if some spare time pops up. A little merging here and there goes a long way!

[JalenOng]

Hey @snipe, @VanillaNinjaD @PromoFaux @Yehonal, @Cervator we are the new team looking into the issue. Since we don't have any test data to work with, can you guys share some repo/tracker links that are not updating? If you can share your first name too, that would be great. Appreciate your privacy, so share whatever that you feel comfortable with. Can also email me at jalen.ong@gmail.com. Thanks!

[snipe]

Hi @JalenOng - no problem. My first name is Alison.

• https://github.com/snipe/snipe-it - repo
• https://www.bountysource.com/teams/snipe-it - BS link

Edit: I removed my original issue link, since that issue was so old, the BS integration was actually working back then. Any newer issue in the repo shows the BS integration not working though. No "Want to back this issue? Post a bounty on it! We accept bounties via Bountysource." on any issues in the few year or so.

[Yehonal]

Hi @JalenOng , Yehonal Here and nice to meet you! I hope bountysource can be boosted with this new team :)

https://github.com/azerothcore/azerothcore-wotlk (repo) https://www.bountysource.com/teams/azerothcore/issues (bountysource)

Currently i'm manually adding the badge to the issues because it doesn't happen automatically

2018-03-02 23:09 GMT+01:00 snipe notifications@github.com:

Hi @JalenOng https://github.com/jalenong - no problem. My first name is Alison.

• https://github.com/snipe/snipe-it - repo
• https://www.bountysource.com/teams/snipe-it - BS link
• https://www.bountysource.com/issues/26718575-feature- request-insurance-for-equipment https://www.bountysource.com/issues/26718575-feature-request-insurance-for-equipment
• example bounty'd issue
• snipe/snipe-it#1176 https://github.com/snipe/snipe-it/issues/1176
• example original issue with no BS integration

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bountysource/core/issues/1096#issuecomment-370067168, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI-lABTIBODxirNPOQWf2EmYv3fyr7Qks5tacMZgaJpZM4MomUa .

[VanillaNinjaD]

Thanks @JalenOng!

My first name is Cole. I’ll be the first to admit that while i’m an IT professional and a full-time tinkerer I am not a developer.

I’ll keep my eyes peeled for absolutely any way that I can help though!

[JalenOng]

@snipe @Yehonal, really appreciate the data. Thank you. Will update soon!

@VanillaNinjaD Thank you! Aren't they all just labels :)

[CendioOssman]

Glad to see this progressing and #1130 merged. :)

Does this mean that the issue is fixed and we'll be getting proper updates?

And what happens to all existing entries in our repos? Will they be adjusted automatically, or do we need to do something?

[youjingwong]

You'll need to unlock repos that aren't updating.

Go to https://www.bountysource.com/tools

You'll find that the project that's not updating to be locked (Let me and @JalenOng know if it isn't!). Just uncheck 'This project is locked' and click save. If all is well, all the issues within that repo should be updated within a few minutes.

Let us know if it works for you :)

[CendioOssman]

Great, thanks. That worked perfectly. :)

[PromoFaux]

@NoNonsense126, One of our repos just goes back to locked again after I uncheck it and then click save... any thoughts?

edit:

 This plugin is locked. 
Reason: undefined method `[]' for nil:NilClass 

[youjingwong]

What's the repo name?

[PromoFaux]

pi-hole/pi-hole

[youjingwong]

@PromoFaux It should be fixed now.

[PromoFaux]

Yeah, just saw all the webhooks coming through for that repo.. thanks!

[Flaburgan]

@NoNonsense126 https://www.bountysource.com/tools doesn't list diaspora/diaspora even if I'm admin of that team on bounty source and a member of the organization

[youjingwong]

@Flaburgan You need to be a public member of the organization

[snipe]

@JalenOng @NoNonsense126 it worked - thanks so much!!

[Flaburgan]

@NoNonsense126 what's the difference with the link I posted? (As you can see I appear in people).

[youjingwong]

@Flaburgan No, I don't see you in the link you shared.

Perhaps you can try it in incognito mode/ when you're not logged in?

[Flaburgan]

Alright I just set my visibility from private to public sorry. It still doesn't appear at the moment but I guess I should wait a bit. However this makes me wondering how the rights check is done in bounty source, because I already granted access to my account so you should be able to check that I have write access to the repository using the github API. Are you manually parsing the people page of the organization? I guess not but eh :p

[Cervator]

Awesome to see this get merged :-)

For a general note / reminder: this current integration is very old and predates the introduction of the new more full GitHub applications. I'd love to just see this one stabilized then eventually introduce a full replacement that has all the bells and whistles

[PromoFaux]

@NoNonsense126 ... How can I completely remove the integration from a repo? The only option I can see on the site is to just untick it's permissions.