Support authn and authz for Bitbucket Server

[beyang]

Feature request description

Support authentication and repository authorization enforcement via Bitbucket Server. This is analogous to the way we support authn/authz for GitHub and GitLab.

This would enable Sourcegraph users to sign in with their Bitbucket Server accounts. Thereafter, the Bitbucket Server user would be associated with their Sourcegraph user and this will be used to enforce repository permissions (as defined by Bitbucket Server).

Implementation questions / customer asks

A wrinkle in this is that Bitbucket Server doesn't yet support OAuth2, which is the favored authentication mechanism for GitHub and GitLab. BBS does support OAuth1, but via Atlassian "application links". The broader question here is whether BBS should be treated as an identity provider at all or if something like Atlassian Crowd (or another SSO provider) should be used. If BBS is not the identity provider, then we'll need to map the SSO identity to the BBS user identity somehow (probably through the BBS API, if it supports user lookup by external ID). The actual permissions lookup can probably be done with this endpoint: https://confluence.atlassian.com/bitbucketserverkb/how-to-report-on-permissions-by-using-rest-api-endpoints-858696596.html.

The ask for customers that use BBS is to describe your current authentication setup and needs. Specifically,

• What is the source of truth for SSO / user identity within your organization? (e.g., Okta, Atlassian Crowd, LDAP, OneLogin, Auth0, Google Auth, or something else?)
• How do you log into BBS? If not via the identity source of truth specified above, please describe.
• Do you have repository permissions that you'd like to enforce on Sourcegraph? What is the source of truth for repository permissions? If it's not BBS, please describe.

Customers and orgs who have a need for Bitbucket permissions:

• https://app.hubspot.com/contacts/2762526/company/434427573/
• https://app.hubspot.com/contacts/2762526/company/557692615/
• https://app.hubspot.com/contacts/2762526/company/557889642/
• https://app.hubspot.com/contacts/2762526/company/407948923/
• https://app.hubspot.com/contacts/2762526/company/1006662755/
• https://app.hubspot.com/contacts/2762526/company/814799452/
• https://app.hubspot.com/contacts/2762526/company/623971957/

[huasome]

• I'm not too familiar with how OAuth integration works in Bitbucket Server. Atlassian doesn't really provide any documentation on it (other than with other Atlassian products) but https://confluence.atlassian.com/applinks/link-atlassian-applications-to-work-together-785449117.html might be a good starting point.

• Most Bitbucket Server instances will use LDAP or AD since they are internal enterprise instances.

• Data Center Edition supports SAML which provides 2FA capabilities. REST API will use basic auth with username/password pairs.

• If the user has read (or above) on the repository, then they should have search permissions on the repository. Note that the permissions are hierachical, global -> probject -> repo.

You may have do periodic polls and cache the permissions for each user on each repo. I can't think of a better way to do it off the top of my head.

I think you can safely assume for most on-prem installs, that the usernames will match across Sourcegraph and Bitbucket.

[Beyang edit] internal discussion thread: https://sourcegraph.slack.com/archives/C0W2E592M/p1543530930126100

[beyang]

Based on the additional feedback in the Slack thread, it looks like username equivalency between the LDAP user and the Bitbucket user is a safe assumption in this case, which suggests a clear implementation path (adding a useUsername field to the hypothetical BitbucketConnection.Authorization config). Awesome.

@huasome one additional question: you indicated that LDAP is the source of identity truth, but the actual sign-in to Bitbucket Server is via SAML. Would it be acceptable (and perhaps even desirable) to use SAML sign-in for Sourcegraph, as well, instead of LDAP directly? The setup I'm envisioning is the following:

• Use SAML to sign into both Bitbucket Server and Sourcegraph
• There is username equivalency between the SAML IdP username and Bitbucket username
• Sourcegraph queries the Bitbucket Server API to determine user permissions, relying on the username equivalency assumption in the previous point

[huasome]

Yes, I do plan to switch Sourcegraph over to use SAML from Google OAuth. However, that I'm a little concerned about getting locked out of admin accounts if I screw it up.

Does Sourcegraph have a backdoor where we can use the local admin account instead of going through the OAuth/SAML login flow?

On Thu, Nov 29, 2018 at 3:38 PM Beyang Liu notifications@github.com wrote:

Based on the additional feedback in the Slack thread, it looks like username equivalency between the LDAP user and the Bitbucket user is a safe assumption in this case, which suggests a clear implementation path (adding a useUsername field to the hypothetical BitbucketConnection.Authorization config). Awesome.

@huasome https://github.com/huasome one additional question: you indicated that LDAP is the source of identity truth, but the actual sign-in to Bitbucket Server is via SAML. Would it be acceptable (and perhaps even desirable) to use SAML sign-in for Sourcegraph, as well, instead of LDAP directly? The setup I'm envisioning is the following:

• Use SAML to sign into both Bitbucket Server and Sourcegraph
• There is username equivalency between the SAML IdP username and Bitbucket username
• Sourcegraph queries the Bitbucket Server API to determine user permissions, relying on the username equivalency assumption in the previous point

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sourcegraph/sourcegraph/issues/1108#issuecomment-443034900, or mute the thread https://github.com/notifications/unsubscribe-auth/ACWmTYTD5kZsXngSVuhrzHNaypTxL0cCks5u0HAIgaJpZM4Yud5b .

[slimsag]

We are working on a management console UI, which will be a separate service running on a separate port with a separate UX. It will have basic authentication, and the goal is that all Sourcegraph settings that could accidentally break Sourcegraph or lock you out as an admin will be configured through that UX, and as such you will always be able to get in and fix them easily. Tracking issue for this: https://github.com/sourcegraph/sourcegraph/issues/965

@beyang I presume I am correct about OAuth/SAML living as part of the management console.

[huasome]

Yes, using SAML to authenticated to Sourcegraph is on our road map.

[beyang]

@slimsag I believe the management console should not be behind OAuth/SAML but just behind its own basic auth (in the same way GitHub Enterprise's management console goes through a completely separate auth mechanism than the application proper). (And customers could choose whether or not to expose its port publicly.) Is that right?

Either way, it can effectively as a backdoor to Sourcegraph if an admin gets "locked out" for some reason.

[slimsag]

@beyang Yes, exactly, that is what I was trying to communicate with my last message. I meant that a TODO here is to ensure that BitBucket server auth is configured via the management console (not all site config will be, only "critical" parts) to ensure that a locked out admin could fix the issue from there. I'll discuss more details on this soon with you.

[keegancsmith]

@sqs Does auth fall under the responsibilities of repo-updater / things on myself and Tomas' plate at the moment? Or is this a few months down the line sort of work?

[sqs]

This is at least a few months away. If prioritized, it probably would fall under you and @tsenart.

[KattMingMing]

Another customer has asked for this – it's currently blocking their full deployment. @sqs, any updates on prioritization here?

[sqs]

@beyang @tsenart @keegancsmith Can you give an estimate for how long this would take and when you could get started on it (if it and #2715 were the next things to do once the current #2025 work is done)? @beyang can help determine this.

@KattMingMing Can you add the HubSpot URLs of all customers that need this to the top issue comment?

[beyang]

I think it would be helpful to chat about the specific customers. The important open questions (many of which are taken from the ACLs questionnaire) are:

• What authentication mechanisms need to be supported? Sign in through BbS or another SSO provider?
• Can we assume that the username on Sourcegraph is the same as the username on Bitbucket Server?

[KattMingMing]

The feedback I've heard from users/customers has been that they would like Bitbucket auth the same way that GitHub auth is supported. Several have LDAP groups for controlling access to signing into Bitbucket so signing in via the code host would accomplish repo permissions and controlling access to their Sourcegraph instance.

It sounds reasonable to disable username changes and use the Bitbucket usernames on the Sourcegraph instance if that would be easier to implement. Also the consistency would be helpful for admins to know which users are accessing their Sourcegraph instance since the usernames will be the same.

[beyang]

This should be doable within the span of 1 iteration. It looks like there will be some extra work required to add OAuth 1 handlers (since AFAICT BbS doesn't support OAuth 2 yet). Does that granularity of estimate suffice? Finer grained will take more looking into.

[KattMingMing]

Sounds great, thanks for the updates and prioritizing!

[sqs]

I am tentatively adding this to the 3.4 triage board with the tentative label so we are reminded to discuss it. Repo perms are coming up more and more in customer requests, and after #2025 is done (and all tech debt and followups are done), it and https://github.com/sourcegraph/sourcegraph/issues/462 are the highest priority things.

[slimsag]

Some more signal working on this makes sense: https://sourcegraph.slack.com/archives/C0W2E592M/p1554865157026700

[tsenart]

This should be doable within the span of 1 iteration. It looks like there will be some extra work required to add OAuth 1 handlers (since AFAICT BbS doesn't support OAuth 2 yet). Does that granularity of estimate suffice? Finer grained will take more looking into.

@beyang: It'd be super helpful if you could summarize the implementation steps and constraints you had in mind.

[beyang]

@tsenart Wrote up my thoughts here, happy to discuss live: https://docs.google.com/document/d/1RW6p7MsVq1lLQ_Ue3wtVPGOIimftVQyY3Fh_x2LZfVk/edit#