Error when integrating Keycloak as a IDP as per https://github.com/sameersbn/docker-gitlab/blob/master/docs/keycloak-idp.md

nemonik commented 3 years ago

@BartJoris for sameersbn/gitlab:13.3.4 and following https://github.com/sameersbn/docker-gitlab/blob/master/docs/keycloak-idp.md guidance I am seeing the following error on start up. This occurs as soon as I add the OUTH2_GENERIC_APP_ID:

Missing Rails.application.secrets.openid_connect_signing_key for production environment. The secret will be generated and stored in config/secrets.yml.
2021-01-06 17:08:57,103 INFO exited: sidekiq (exit status 1; not expected)
2021-01-06 17:08:57,430 INFO spawned: 'sidekiq' with pid 787
2021-01-06 17:08:57,470 INFO exited: puma (exit status 1; not expected)
2021-01-06 17:08:58,473 INFO spawned: 'puma' with pid 788
2021-01-06 17:08:58,474 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-06 17:08:59,475 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
rake aborted!
ArgumentError: Missing :action key on routes definition, please check your routes.
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:337:in `check_part'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:326:in `check_controller_and_action'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:262:in `normalize_options!'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:131:in `initialize'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:83:in `new'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:83:in `build'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1955:in `add_route'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1927:in `decomposed_match'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1891:in `block in map_match'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1885:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1885:in `map_match'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1633:in `match'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:452:in `block in devise_omniauth_callback'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:446:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:446:in `devise_omniauth_callback'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:268:in `block (4 levels) in devise_for'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:268:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:268:in `block (3 levels) in devise_for'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:472:in `with_devise_exclusive_scope'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:267:in `block (2 levels) in devise_for'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:370:in `block in devise_scope'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1016:in `block in constraints'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:887:in `scope'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/mapper.rb:1016:in `constraints'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:369:in `devise_scope'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:266:in `block in devise_for'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:242:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/devise-4.7.1/lib/devise/rails/routes.rb:242:in `devise_for'
(eval):21:in `draw_route'
/home/git/gitlab/lib/gitlab/patch/draw_route.rb:30:in `instance_eval'
/home/git/gitlab/lib/gitlab/patch/draw_route.rb:30:in `draw_route'
/home/git/gitlab/lib/gitlab/patch/draw_route.rb:17:in `draw_ce'
/home/git/gitlab/lib/gitlab/patch/draw_route.rb:11:in `draw'
/home/git/gitlab/config/routes.rb:271:in `block in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:426:in `instance_exec'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:426:in `eval_block'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.3.1/lib/action_dispatch/routing/route_set.rb:408:in `draw'
/home/git/gitlab/config/routes.rb:5:in `<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:318:in `load'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:318:in `block in load'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:291:in `load_dependency'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:318:in `load'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application/routes_reloader.rb:40:in `block in load_paths'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application/routes_reloader.rb:40:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application/routes_reloader.rb:40:in `load_paths'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application/routes_reloader.rb:20:in `reload!'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application.rb:169:in `reload_routes!'
/home/git/gitlab/config/application.rb:319:in `block in <class:Application>'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:68:in `block in execute_hook'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:61:in `with_execution_control'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:66:in `execute_hook'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:52:in `block in run_load_hooks'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:51:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/lazy_load_hooks.rb:51:in `run_load_hooks'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application/finisher.rb:129:in `block in <module:Finisher>'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:32:in `instance_exec'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:32:in `run'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:61:in `block in run_initializers'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/initializable.rb:60:in `run_initializers'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application.rb:363:in `initialize!'
/home/git/gitlab/config/environment.rb:5:in `<top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324:in `require'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324:in `block in require'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:291:in `load_dependency'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.1/lib/active_support/dependencies.rb:324:in `require'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application.rb:339:in `require_environment!'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.1/lib/rails/application.rb:523:in `block in run_tasks_blocks'
/home/git/gitlab/vendor/bundle/ruby/2.6.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => gitlab:setup => gitlab_environment => environment
(See full trace by running task with --trace)

I am providing

        - name: OAUTH_ENABLED
          value: "true"
        - name: OAUTH_AUTO_SIGN_IN_WITH_PROVIDER
          value: "Keycloak"
        - name: OAUTH_ALLOW_SSO
          value: "Keycloak"
        - name: OAUTH_BLOCK_AUTO_CREATED_USERS
          value: "false"
        - name: OAUTH_AUTO_LINK_LDAP_USER
          value: "false"
        - name: OAUTH_AUTO_LINK_SAML_USER
          value: "false"
        - name: OAUTH_EXTERNAL_PROVIDERS
          value: "Keycloak"

        - name: OAUTH2_GENERIC_APP_ID
          value: "gitlab_client_id_in_keycloak"
        - name: OAUTH2_GENERIC_APP_SECRET
          value: "secret out of keycloak for client"
        - name: OAUTH2_GENERIC_CLIENT_SITE
          value: "https://keycloak.example.com"
        - name: OAUTH2_GENERIC_CLIENT_USER_INFO_URL
          value: "https://keycloak.example.com/auth/realms/example.com/protocol/openid-connect/userinfo"
        - name: OAUTH2_GENERIC_CLIENT_AUTHORIZE_URL
          value: "https://keycloak.example.com/auth/realms/example.com/protocol/openid-connect/auth"
        - name: OAUTH2_GENERIC_CLIENT_TOKEN_URL
          value: "https://keycloak.example.com/auth/realms/example.com/protocol/openid-connect/token"
        - name: OAUTH2_GENERIC_CLIENT_END_SESSION_ENDPOINT
          value: "https://keycloak.example.com/auth/realms/example.com/protocol/openid-connect/logout"

Does a referenced docker-compose exist for what the document describes? Is it an additional configuration item? Something breaking in this release?

nemonik commented 3 years ago

I think not documenting the need to set OAUTH2_GENERIC_NAME maybe the problem.

- name: OAUTH2_GENERIC_NAME
  value: Keycloak

nemonik commented 3 years ago

And you have to configure at least for me... as per Keycloak's Client Scopes.

        - name: OAUTH2_GENERIC_USER_UID
          value: "preferred_username"
        - name: OAUTH2_GENERIC_USER_NAME
          value: "name"
        - name: OAUTH2_GENERIC_USER_EMAIL
          value: "email"

nemonik commented 3 years ago

I'll propose some edits to the doc... in a day or two... in a pull request.

nemonik commented 3 years ago

Do not set OAUTH_EXTERNAL_PROVIDERS=Keycloak as shown in the doc as your users will not be able to create projects... Their project limit will default to 0.

Use

        - name: OAUTH_EXTERNAL_PROVIDERS
          value: ""

nemonik commented 3 years ago

I authored a pull request addressing https://github.com/sameersbn/docker-gitlab/pull/2293

Kristaba commented 3 years ago

Thanks a lot @nemonik for improving this part of the documentation!

I also encounter an error (both sidekiq and puma exit with error code 1, with no log apparently) as soon as I set OAUTH2_GENERIC_APP_ID. And curiously happens even if I set OAUTH_ENABLED=false.

Is there a workaround or something I can do to debug this issue?

benedikt-bartscher commented 2 years ago

@Kristaba, i have the same issue. Without OAUTH2_GENERIC_APP_ID i get an error 500 on login page, with it sidekiq and puma are stuck in a restart loop.

benedikt-bartscher commented 2 years ago

@Kristaba maybe this is overwriting your OAUTH_ENABLED?:

Enable OAuth support. Defaults to true if any of the support OAuth providers is configured, else defaults to false.

nhh commented 2 years ago

I some sort of this problem too, I configured my gitlab according to https://github.com/sameersbn/docker-gitlab/blob/master/docs/keycloak-idp.md

And there is no login button for Keycloak showing up.

I am using docker image version sameersbn/gitlab:14.9.3 and this is my OAUTH config:

    - OAUTH_ENABLED=true
    - OAUTH_ALLOW_SSO=Keycloak
    - OAUTH_BLOCK_AUTO_CREATED_USERS=false
    - OAUTH_AUTO_LINK_LDAP_USER=false
    - OAUTH_AUTO_LINK_SAML_USER=false

    - OAUTH2_GENERIC_APP_SECRET=my_token
    - OAUTH2_GENERIC_CLIENT_SITE=https://auth.example.com
    - OAUTH2_GENERIC_CLIENT_USER_INFO_URL=https://auth.example.com/auth/realms/example/protocol/openid-connect/userinfo
    - OAUTH2_GENERIC_CLIENT_AUTHORIZE_URL=https://auth.example.com/auth/realms/example/protocol/openid-connect/auth
    - OAUTH2_GENERIC_CLIENT_TOKEN_URL=https://auth.example.com/auth/realms/example/protocol/openid-connect/token
    - OAUTH2_GENERIC_CLIENT_END_SESSION_ENDPOINT=https://auth.example.com/auth/realms/example/protocol/openid-connect/logout

    - OAUTH2_GENERIC_USER_UID='preferred_username'
    - OAUTH2_GENERIC_USER_NAME='name'
    - OAUTH2_GENERIC_USER_EMAIL='email'

I even named the client in Keycloak git because the docs says it and I only can configure the ID

Any ideas? Thank you in advance!

EDIT: 15.04.2022

I got it working:

    - OAUTH_ENABLED=true
    - OAUTH_ALLOW_SSO=Keycloak
    - OAUTH_BLOCK_AUTO_CREATED_USERS=false
    - OAUTH_AUTO_LINK_LDAP_USER=false
    - OAUTH_AUTO_LINK_SAML_USER=false

    - OAUTH2_GENERIC_NAME=Keycloak
    - OAUTH2_GENERIC_APP_SECRET=my_token
    - OAUTH2_GENERIC_CLIENT_SITE=https://auth.example.com
    - OAUTH2_GENERIC_CLIENT_USER_INFO_URL=https://auth.example.com/realms/example/protocol/openid-connect/userinfo
    - OAUTH2_GENERIC_CLIENT_AUTHORIZE_URL=https://auth.example.com/realms/example/protocol/openid-connect/auth
    - OAUTH2_GENERIC_CLIENT_TOKEN_URL=https://auth.example.com/realms/example/protocol/openid-connect/token
    - OAUTH2_GENERIC_CLIENT_END_SESSION_ENDPOINT=https://auth.example.com/realms/example/protocol/openid-connect/logout

    - OAUTH2_GENERIC_USER_UID='preferred_username'
    - OAUTH2_GENERIC_USER_NAME='name'
    - OAUTH2_GENERIC_USER_EMAIL='email'

Several things to watch out for:

Sidekiq is restarting in a certain combination (only reinstalling gitlab helped)
You cannot rename OAUTH2_GENERIC_NAME after initially set -> HTTP ERROR 422
The correct configuration sometimes throws Yaml/Psych errors when database is not yet migrated. (just wait until db migrated)
IMPORTANT the keycloak urls are not having /auth as suffix:
- https://auth.example.com/auth/realms... vs. https://auth.example.com/realms...

sameersbn / docker-gitlab

Error when integrating Keycloak as a IDP as per https://github.com/sameersbn/docker-gitlab/blob/master/docs/keycloak-idp.md #2282