Is there a way to link to existing non-google accounts?

[cheshirekow]

We have a gerrit instance running 2.9.4 where our users are currently authenticating using Yahoo OpenID. We would like to switch to using google OAuth. I've tested migration of our instance to gerrit 2.11.2 and installed the Oauth plugin release 2.11. I'm able to login with google, but it creates a new account instead of linking to the current account associated with that email address.

On a fresh clone of our instance, I've also tried using the settings -> identities -> "Link Another Identity" link, but when I get back to gerrit after logging in with google, I'm now signed in as a new user, and not linked to the original user. This doesn't seem releated to FAQ "Why linking of user identities doesn't work from the UI" as it seems that issue was with the link going to an empty page.

Are either of these migration paths intended to work? If not, is there any way to migrate/link existing accounts to use an OAuth identity? (through gerrit command line or other?)

[davido]

The only way that supposed to work in your scenario (coming from OpenId Yahoo) is to use Hybrid OpenId+OAuth2 auth scheme.

• Let auth to be OpenId
• Install the plugin and enable Google OAuth provider
• Login with Yahoo OpenId
• Settings -> identities -> "Link Another Identity"
• Sign in with Google OAuth

Google OAuth account should be linked to the existing Yahoo account. When it doesn't work it's a bug. In this case, try to increase log level and post your log results here (in one issue i explained how to do it).

[cheshirekow]

Ok, that doesn't work. I'll describe what I've tried:

I've got a docker container running a copy of what is currently deployed. From within that docker container I download gerrit-2.11.2.war (as gerrit.war) and download the gerrit-oauth-provider plugin into ~/gerrit/plugins. Here is the remainder of my migration session:

gerrit2@53bb46c39224:~$ java -jar gerrit.war init -d ${HOME}/gerrit
Using secure store: com.google.gerrit.server.securestore.DefaultSecureStore

*** Gerrit Code Review 2.11.2
*** 

*** Git Repositories
*** 

Location of Git repositories   [git]: 

*** SQL Database
*** 

Database server type           [h2]: 

*** Index
*** 

Type                           [LUCENE/?]: 

The index must be rebuilt before starting Gerrit:
  java -jar gerrit.war reindex -d site_path

*** User Authentication
*** 

Authentication method          [OPENID/?]: 

*** Email Delivery
*** 

SMTP server hostname           [smtp-relay.gmail.com]: 
SMTP server port               [(default)]: 
SMTP encryption                [NONE/?]: 
SMTP username                  : 

*** Container Process
*** 

Run as                         [gerrit2]: 
Java runtime                   [/usr/lib/jvm/java-7-openjdk-amd64/jre]: 
Upgrade /home/gerrit2/gerrit/bin/gerrit.war [Y/n]? Y
Copying gerrit.war to /home/gerrit2/gerrit/bin/gerrit.war

*** SSH Daemon
*** 

Listen on address              [*]: 
Listen on port                 [29418]: 

*** HTTP Daemon
*** 

Behind reverse proxy           [y/N]? 
Use SSL (https://)             [Y/n]? 
Listen on address              [*]: 
Listen on port                 [8080]: 
Canonical URL                  [https://<removed>/]: https://localhost:8444
Create new self-signed SSL certificate [y/N]? 

*** Plugins
*** 

Installing plugins.
Install plugin reviewnotes version v2.11.2 [y/N]? 
Install plugin replication version v2.11.2 [y/N]? 
Install plugin download-commands version v2.11.2 [y/N]? 
Install plugin singleusergroup version v2.11.2 [y/N]? 
Install plugin commit-message-length-validator version v2.11.2 [y/N]? 
Initializing plugins.

*** OAuth Authentication Provider
*** 

Use Google OAuth provider for Gerrit login ? [Y/n]? 
Application client id          : <removed>
Application client secret      : 
              confirm password : 
Link to OpenID accounts?       [true]: 
Use GitHub OAuth provider for Gerrit login ? [Y/n]? n

Upgrading schema to 94 ...
Upgrading schema to 95 ...
Upgrading schema to 96 ...
Upgrading schema to 97 ...
Upgrading schema to 98 ...
Upgrading schema to 99 ...
Upgrading schema to 100 ...
Upgrading schema to 101 ...
Upgrading schema to 102 ...
Upgrading schema to 103 ...
Upgrading schema to 104 ...
Upgrading schema to 105 ...
Upgrading schema to 106 ...
Upgrading schema to 107 ...
Migrating data to schema 94 ...
Migrating data to schema 95 ...
Migrating data to schema 96 ...
Migrating data to schema 97 ...
Migrating data to schema 98 ...
Migrate user preference showUserInReview to reviewCategoryStrategy
Migrating data to schema 99 ...
Migrating data to schema 100 ...
Migrating data to schema 101 ...
Migrating data to schema 102 ...
Migrating data to schema 103 ...
Migrating data to schema 104 ...
Migrating data to schema 105 ...
Dropping index changes_byproject on table changes
Dropping index changes_key on table changes
Dropping index changes_submitted on table changes
Dropping index changes_byprojectopen on table changes
Migrating data to schema 106 ...
listing all repositories ...
done
creating reflog files for refs/meta/config branches ...
done
Migrating data to schema 107 ...
Execute the following SQL to drop unused objects:

  ALTER TABLE accounts DROP COLUMN comment_visibility_strategy;
  ALTER TABLE accounts DROP COLUMN reverse_patch_set_order;
  ALTER TABLE accounts DROP COLUMN change_screen;
  ALTER TABLE accounts DROP COLUMN show_user_in_review;
  ALTER TABLE changes DROP COLUMN mergeable;
  ALTER TABLE changes DROP COLUMN last_sha1_merge_tested;
  ALTER TABLE changes DROP COLUMN open;
  ALTER TABLE changes DROP COLUMN sort_key;

Execute now [Y/n]? Y
Initialized /home/gerrit2/gerrit
gerrit2@53bb46c39224:~$ java jar gerrit.war reindex -d ${HOME}/gerrit
Error: Could not find or load main class jar
gerrit2@53bb46c39224:~$ java -jar gerrit.war reindex -d ${HOME}/gerrit
[2015-09-16 15:03:00,558] INFO  com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 889m
[2015-09-16 15:03:00,909] INFO  com.google.gerrit.server.cache.h2.H2CacheFactory : Enabling disk cache /home/gerrit2/gerrit/cache
Collecting projects:    4
Reindexing changes: projects: 100% (4/4), 100% (1053/1053), done    
Reindexed 1053 changes in 854.0s (1.2/s)

gerrit2@53bb46c39224:~$ ./start.sh 
Starting Gerrit Code Review: OK
[2015-09-16 15:27:43,341] INFO  com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 889m
[2015-09-16 15:27:43,351] INFO  com.google.gerrit.server.plugins.PluginLoader : Loading plugins from /home/gerrit2/gerrit/plugins
[2015-09-16 15:27:43,394] INFO  com.google.gerrit.server.plugins.PluginLoader : Loaded plugin gerrit-oauth-provider, version 2.11
[2015-09-16 15:27:43,397] INFO  com.google.gerrit.server.plugins.PluginLoader : Loaded plugin disable_submit_button, version 
[2015-09-16 15:27:43,650] INFO  com.google.gerrit.sshd.SshDaemon : Started Gerrit SSHD-CORE-0.14.0 on *:29418
[2015-09-16 15:27:43,654] INFO  org.eclipse.jetty.server.Server : jetty-9.2.9.v20150224
[2015-09-16 15:27:43,941] INFO  org.eclipse.jetty.server.handler.ContextHandler : Started o.e.j.s.ServletContextHandler@257399ca{/,file:/home/gerrit2/gerrit/tmp/gerrit_3252646714469433030_app/gerrit_war/,AVAILABLE}
[2015-09-16 15:27:43,964] INFO  org.eclipse.jetty.server.ServerConnector : Started ServerConnector@4e9ceaa9{SSL-http/1.1}{0.0.0.0:8080}
[2015-09-16 15:27:43,965] INFO  org.eclipse.jetty.server.Server : Started @4491ms
[2015-09-16 15:27:43,966] INFO  com.google.gerrit.pgm.Daemon : Gerrit Code Review 2.11.2 ready

I then run ssh -p 29418 localhost gerrit logging set-level DEBUG from a shell outside the container to increase logging level. Then I go through the web interface and use Settings -> identities -> "Link Another Identity". I sign in with google, and when it comes back to gerrit I'm logged in as a new user.

Is there anything I can look for in the logs to help figure out what's going on?

[davido]

That's strange. I tested it and it worked. Just to make sure: you login with OpenId Yahoo account and select Settings -> identities -> "Link Another Identity" and login with Google OAuth that you configured over gerrit-oauth-plugin?

This code path should work like this: When LoginForm is activated from Settings -> identities -> "Link Another Identity", then link mode should be passed to LoginForm: [1]. When actual login happens with OAuth identity, then this mode should be retrieved and linking of identity should happen instead of registering new account: [2]. Check the log for any failures? It shouldn't matter which OpenID provider is used, but I've tested it with Launchpad OpenId <-> Google OAuth account linking and it worked.

[1] https://github.com/gerrit-review/gerrit/blob/master/gerrit-openid/src/main/java/com/google/gerrit/httpd/auth/openid/LoginForm.java#L158,L160

[2] https://github.com/gerrit-review/gerrit/blob/master/gerrit-openid/src/main/java/com/google/gerrit/httpd/auth/openid/OAuthSessionOverOpenID.java#L158,L169

[cheshirekow]

Gotcha. The first thing I'll try is gerrit 2.11.3 and plugin 11.3. Then I'll dig through the logs for errors. Here's a youtube video of my testing the process (this is in the docker container, note the url is localhost).

https://youtu.be/yLiF1UKw9RA

Please let me know if you see anything out of place about what I'm doing in the video.

[cheshirekow]

Just a quick note, same behavior on gerrit 2.11.3 and plugin 11.3

[davido]

Before I'll re-check it all again, could you do one more test:

• Set-up Gerrit 2.11.3 site from scratch: no users in the database, dump database user tables to be sure
• Auth scheme OpenId + gerrit-oauth-plugin with Google OAuth provider
• First login with OpenId Yahoo (or Launchpad) provider: dump the database record for this user
• Settings -> identities -> "Link Another Identity" -> Register with Google OAuth: dump the database record for this user

[cheshirekow]

From a fresh container (I don't know how to dump the database except through gerrit gsql and I can't do that without a user yet...).

1. Login with yahoo Note: on return to gerrit it says "Not Found", "The page you requested was not found, or you do not have permission to view this page". the url on return is https://localhost:8444/#/register#/q/status:open
2. Edit All-Projects project, under Access, click 'Edit' and add permission 'Access database' for the 'Administrator' group
3. Save changes
4. In user -> Settings -> ssh public keys, add public key
5. In user -> Settings -> profile, select username
6. From a terminal run ssh -p 29418 localhost gerrit gsql

gerrit> SELECT * FROM ACCOUNTS;
 REGISTERED_ON           | FULL_NAME | PREFERRED_EMAIL | CONTACT_FILED_ON | MAXIMUM_PAGE_SIZE | SHOW_SITE_HEADER | USE_FLASH_CLIPBOARD | DOWNLOAD_URL | DOWNLOAD_COMMAND | COPY_SELF_ON_EMAIL | DATE_FORMAT | TIME_FORMAT | RELATIVE_DATE_IN_CHANGE_TABLE | DIFF_VIEW | SIZE_BAR_IN_CHANGE_TABLE | LEGACYCID_IN_CHANGE_TABLE | REVIEW_CATEGORY_STRATEGY | MUTE_COMMON_PATH_PREFIXES | INACTIVE | ACCOUNT_ID
 ------------------------+-----------+-----------------+------------------+-------------------+------------------+---------------------+--------------+------------------+--------------------+-------------+-------------+-------------------------------+-----------+--------------------------+---------------------------+--------------------------+---------------------------+----------+-----------
 2015-09-18 15:48:33.136 | NULL      | NULL            | NULL             | 25                | Y                | Y                   | NULL         | NULL             | N                  | NULL        | NULL        | N                             | NULL      | Y                        | N                         | NULL                     | Y                         | N        | 1000000
(1 row; 8 ms)

1. 5. In user -> Settings -> Idenitites, Link Another Identity
2. Click Google OAuth2 (gerrit-oauth-provider plugin)

gerrit> SELECT * FROM ACCOUNTS;
 REGISTERED_ON           | FULL_NAME       | PREFERRED_EMAIL | CONTACT_FILED_ON | MAXIMUM_PAGE_SIZE | SHOW_SITE_HEADER | USE_FLASH_CLIPBOARD | DOWNLOAD_URL | DOWNLOAD_COMMAND | COPY_SELF_ON_EMAIL | DATE_FORMAT | TIME_FORMAT | RELATIVE_DATE_IN_CHANGE_TABLE | DIFF_VIEW | SIZE_BAR_IN_CHANGE_TABLE | LEGACYCID_IN_CHANGE_TABLE | REVIEW_CATEGORY_STRATEGY | MUTE_COMMON_PATH_PREFIXES | INACTIVE | ACCOUNT_ID
 ------------------------+-----------------+-----------------+------------------+-------------------+------------------+---------------------+--------------+------------------+--------------------+-------------+-------------+-------------------------------+-----------+--------------------------+---------------------------+--------------------------+---------------------------+----------+-----------
 2015-09-18 15:48:33.136 | NULL            | NULL            | NULL             | 25                | Y                | Y                   | NULL         | NULL             | N                  | NULL        | NULL        | N                             | NULL      | Y                        | N                         | NULL                     | Y                         | N        | 1000000
 2015-09-18 15:54:20.891 | Josh Bialkowski | josh@skyd.io    | NULL             | 25                | Y                | Y                   | NULL         | NULL             | N                  | NULL        | NULL        | N                             | NULL      | Y                        | N                         | NULL                     | Y                         | N        | 1000001
(2 rows; 5 ms)

[cheshirekow]

Note: It looks like if I manually modify the ACCOUNT_EXTERNAL_IDS table to change the ACCOUNT_ID of the new entry, then future logins with google will log me in as the first user, not the second. Also, it looks like the ACCOUNT_EXTERNAL_IDS entry for my new user contains the google id (i.e. the one you can see in the URL on google+). Perhaps I can just manually add entries to the ACCOUNT_EXTERNAL_IDS table using everyone's google id?

[davido]

Sure you could do that, but say your site has >1K users, you have better plan for the next time than migrate all them manually, right? I'm looking into it. Now.

[davido]

As expected, I'm unable to reproduce the problem you are describing. Works here as expected:

gerrit.config:

[auth]
    type = OPENID
    trustedOpenID=^.*$

• Gerrit: master
• Plugin: master
• Create vanilla site and login with Launchpad OpenId identity (I don't have Yahoo)
• Add view database capability
• Set username
• Add SSH public key
• Database dump: (ssh host gerrit gsql) admin user is created
• Configure gerrit-oauth-plugin, with Google OAuth provider only
• Restart Gerrit
• Login with Launchpad, Settings -> Identities -> Link Another Identity
• Register with OAuth Gerrit provider
• I'm back to the Identites screen, with two rows in the table: Launchpad + Google OAuth
• In short, new OAuth identity is linked to the same admin account
• Dump database:

gerrit> select * from ACCOUNT_EXTERNAL_IDS;
 ACCOUNT_ID | EMAIL_ADDRESS        | PASSWORD | EXTERNAL_ID
 -----------+---------------------------+----------+----------------------------------
 1000000    | joe.dow@example.com    | NULL   | https://login.launchpad.net/+id/adJDsZ
 1000000    | NULL                   | NULL   | username:joedow
 1000000    | joe.dow@gmail.com      | NULL   | 34534523945376523984
(3 rows; 2 ms)

[davido]

I've uploaded this change: [1] and added vebose output to troubleshoot this issue: [1].

Here is my output on the vanilla Gerrit site, that corresponds to my previous comment. Apply this patch on your Gerrit tree, build it and try to create the same output on vanilla Gerrit site.

Signing in phase with OpenID provider:

DEBUG com.google.gerrit.httpd.auth.openid.LoginForm : mode "SIGN_IN"
DEBUG com.google.gerrit.httpd.auth.openid.LoginForm : OpenId provider
    "https://login.launchpad.net/+openid"
DEBUG com.google.gerrit.httpd.auth.openid.OpenIdServiceImpl : OpenID:
openid-realm=http://localhost:8080/

Following by Identity linking phase with OAuth provider:

DEBUG com.google.gerrit.httpd.auth.openid.LoginForm : mode
    "LINK_IDENTIY"
DEBUG com.google.gerrit.httpd.auth.openid.LoginForm : OAuth provider
    "http://gerrit-oauth-provider_-google-oauth"
[...]
DEBUG com.google.gerrit.httpd.auth.openid.OAuthSessionOverOpenID :
    Linking "34534523945376523984" to "1000000"

• [1] https://gerrit-review.googlesource.com/71041

[cheshirekow]

Ok, I've cherry picked your change into the release-2.11, built and tried again. I can see in the log where the "LINK_IDENTITY" starts:

[2015-09-23 15:54:54,281] DEBUG com.google.gerrit.httpd.auth.openid.LoginForm : mode "LINK_IDENTIY"
[2015-09-23 15:54:54,281] DEBUG com.google.gerrit.httpd.auth.openid.LoginForm : OAuth provider "http://gerrit-oauth-provider_-google-oauth"

But I never see "Linking" in the log. I can see where the oauth comes back and queries my user info from google:

[2015-09-23 15:54:55,108] DEBUG com.googlesource.gerrit.plugins.oauth.GoogleOAuthService : User info response: {
...
}
[2015-09-23 15:54:55,109] DEBUG com.googlesource.gerrit.plugins.oauth.GoogleOAuthService : OAuth2: openid_id=https://www.google.com/accounts/o8/id?id=AItOawmHiNxTFMJo6alF39gOnHswEPD_-Pa07Z0

but I do not see any further entries from com.google.gerrit.httpd.auth.*.

Would you like a paste of the log from "LINK_IDENTITY" to the end?