search

apache / superset

Apache Superset is a Data Visualization and Data Exploration Platform
https://superset.apache.org/
Apache License 2.0
63.13k stars 13.99k forks source link

Use keycloak as OIDC/IdP provider but cannot access to users list #19643

Open PatBriPerso opened 2 years ago

PatBriPerso commented 2 years ago

I setup superset to authenticate the users with keycloak as IdP. I'm able to connect to superset after validating a login/pwd on keycloak but I cannot access the users list page (/users/list) as an Admin.

How to reproduce the bug

Setup superset as described in "Additional context". Connect to superset through keycloak as an Admin Click on the menu Settings > List Users

Expected results

See the users list

Actual results

I'm back on the welcome page (/superset/welcome/) with a message saying "Access is Denied".

Screenshots

n/a

Environment

(please complete the following information):

Checklist

Make sure to follow these steps before submitting your issue - thank you!

Additional context

I have a keycloak setup with the url: https://auth.mydomain.com/ I create a realm named "demo" and a user on this realm. I add a client named "superset" (client ID) on this realm with a Client Protocol "openid-connect" and a Root URL "https://superset.demo.mydomain.com/"

My superset is accessible with the url: https://superset.demo.mydomain.com/

I use the Docker version of superset deployed on a Docker Swarm cluster. I use Traefik to route the HTTP requests to the containers (superset and keycloak).

To setup superset with keycloak, I follow these posts:

But I modify some files so I describe below my whole configuration.

Content of /app/docker/requirements-local.txt:

clickhouse-driver>=0.2.0
clickhouse-sqlalchemy>=0.1.6,<0.2.0
mysql-connector-python
flask-oidc==1.3.0

The first 2 packages are used to connect to my clickhouse database. The third one is used to have the superset database on MySQL (instead of postgres). Only the fourth one is related to keycloak to enable OIDC (OpenID Connect).

Content of /app/pythonpath/superset_config.py:

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#
# This file is included in the final Docker image and SHOULD be overridden when
# deploying the image to prod. Settings configured here are intended for use in local
# development environments. Also note that superset_config_docker.py is imported
# as a final step as a means to override "defaults" configured here
#
import logging
import os
from datetime import timedelta
from typing import Optional

from cachelib.file import FileSystemCache
from celery.schedules import crontab

logger = logging.getLogger()

def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
    """Get the environment variable or raise exception."""
    try:
        return os.environ[var_name]
    except KeyError:
        if default is not None:
            return default
        else:
            error_msg = "The environment variable {} was missing, abort...".format(
                var_name
            )
            raise EnvironmentError(error_msg)

DATABASE_DIALECT = get_env_variable("DATABASE_DIALECT")
DATABASE_USER = get_env_variable("DATABASE_USER")
DATABASE_PASSWORD = get_env_variable("DATABASE_PASSWORD")
DATABASE_HOST = get_env_variable("DATABASE_HOST")
DATABASE_PORT = get_env_variable("DATABASE_PORT")
DATABASE_DB = get_env_variable("DATABASE_DB")

# The SQLAlchemy connection string.
SQLALCHEMY_DATABASE_URI = "%s://%s:%s@%s:%s/%s" % (
    DATABASE_DIALECT,
    DATABASE_USER,
    DATABASE_PASSWORD,
    DATABASE_HOST,
    DATABASE_PORT,
    DATABASE_DB,
)

REDIS_HOST = get_env_variable("REDIS_HOST")
REDIS_PORT = get_env_variable("REDIS_PORT")
REDIS_CELERY_DB = get_env_variable("REDIS_CELERY_DB", "0")
REDIS_RESULTS_DB = get_env_variable("REDIS_RESULTS_DB", "1")

##### Change from original file
#RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")
# Put results on Redis not on a file that is not shared among container on Swarm stack
from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(host=REDIS_HOST, port=REDIS_PORT, key_prefix='superset_results')
##### End of change

class CeleryConfig(object):
    BROKER_URL = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
    CELERY_RESULT_BACKEND = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
    CELERYD_LOG_LEVEL = "DEBUG"
    CELERYD_PREFETCH_MULTIPLIER = 1
    CELERY_ACKS_LATE = False
    CELERYBEAT_SCHEDULE = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=10, hour=0),
        },
    }

CELERY_CONFIG = CeleryConfig

FEATURE_FLAGS = {"ALERT_REPORTS": True}
ALERT_REPORTS_NOTIFICATION_DRY_RUN = True
WEBDRIVER_BASEURL = "http://superset:8088/"
# The base URL for the email report hyperlinks.
WEBDRIVER_BASEURL_USER_FRIENDLY = WEBDRIVER_BASEURL

SQLLAB_CTAS_NO_LIMIT = True

#
# Optionally import superset_config_docker.py (which will have been included on
# the PYTHONPATH) in order to allow for local settings to be overridden
#
try:
    import superset_config_docker
    from superset_config_docker import *  # noqa

    logger.info(
        f"Loaded your Docker configuration at " f"[{superset_config_docker.__file__}]"
    )
except ImportError:
    logger.info("Using default Docker config...")

Content of /app/pythonpath/superset_config_docker.py:

import os
from typing import Optional

def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
    """Get the environment variable or raise exception."""
    try:
        return os.environ[var_name]
    except KeyError:
        if default is not None:
            return default
        else:
            error_msg = "The environment variable {} was missing, abort...".format(
                var_name
            )
            raise EnvironmentError(error_msg)

# The allowed translation for you app
LANGUAGES = {
    "en": {"flag": "us", "name": "English"},
    #"es": {"flag": "es", "name": "Spanish"},
    #"it": {"flag": "it", "name": "Italian"},
    "fr": {"flag": "fr", "name": "French"},
    #"zh": {"flag": "cn", "name": "Chinese"},
    #"ja": {"flag": "jp", "name": "Japanese"},
    #"de": {"flag": "de", "name": "German"},
    #"pt": {"flag": "pt", "name": "Portuguese"},
    #"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
    #"ru": {"flag": "ru", "name": "Russian"},
    #"ko": {"flag": "kr", "name": "Korean"},
    #"sk": {"flag": "sk", "name": "Slovak"},
    #"sl": {"flag": "si", "name": "Slovenian"},
}

ENABLE_PROXY_FIX = True

#---------------------------KEYCLOACK ----------------------------
# See: https://github.com/apache/superset/discussions/13915
# See: https://stackoverflow.com/questions/54010314/using-keycloakopenid-connect-with-apache-superset/54024394#54024394

from keycloak_security_manager  import  OIDCSecurityManager
from flask_appbuilder.security.manager import AUTH_OID

OIDC_ENABLE = get_env_variable("OIDC_ENABLE", 'False')

if OIDC_ENABLE == 'True':
    AUTH_TYPE = AUTH_OID
    SECRET_KEY = get_env_variable("SECRET_KEY", 'ChangeThisKeyPlease')
    OIDC_CLIENT_SECRETS = get_env_variable("OIDC_CLIENT_SECRETS", '/app/pythonpath/client_secret.json')
    OIDC_ID_TOKEN_COOKIE_SECURE = False
    OIDC_REQUIRE_VERIFIED_EMAIL = False
    OIDC_OPENID_REALM = get_env_variable("OIDC_OPENID_REALM")
    OIDC_INTROSPECTION_AUTH_METHOD = 'client_secret_post'
    CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
    AUTH_USER_REGISTRATION = True
    AUTH_USER_REGISTRATION_ROLE = get_env_variable("AUTH_USER_REGISTRATION_ROLE", 'Admin')
#--------------------------------------------------------------

Content of /app/pythonpath/keycloak_security_manager.py:

from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging
from flask import redirect, request

class OIDCSecurityManager(SupersetSecurityManager):

    def __init__(self, appbuilder):
        super(OIDCSecurityManager, self).__init__(appbuilder)
        if self.auth_type == AUTH_OID:
            self.oid = OpenIDConnect(self.appbuilder.get_app)
        self.authoidview = AuthOIDCView

class AuthOIDCView(AuthOIDView):

    @expose('/login/', methods=['GET', 'POST'])
    def login(self, flag=True):
        sm = self.appbuilder.sm
        oidc = sm.oid

        @self.appbuilder.sm.oid.require_login
        def handle_login():
            user = sm.auth_user_oid(oidc.user_getfield('email'))

            if user is None:
                info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
                user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'),
                                   info.get('email'), sm.find_role('Admin'))

            login_user(user, remember=False)
            return redirect(self.appbuilder.get_url_for_index)

        return handle_login()

    @expose('/logout/', methods=['GET', 'POST'])
    def logout(self):
        oidc = self.appbuilder.sm.oid

        oidc.logout()
        super(AuthOIDCView, self).logout()
        redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login

        return redirect(
            oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))

Content of /app/pythonpath/client_secret.json:

{
  "web": {
      "issuer": "https://auth.mydomain.com/realms/demo",
      "auth_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/auth",
      "client_id": "superset",
      "client_secret": "<Client Secret>",
      "redirect_uris": [
          "https://superset.demo.mydomain.com/*"
      ],
      "userinfo_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/userinfo",
      "token_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/token",
      "token_introspection_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/token/introspect"
  }
}

The environment variables I pass to my Docker Swarm service:

FLASK_ENV=production
SUPERSET_ENV=production
SUPERSET_LOAD_EXAMPLES=no
CYPRESS_CONFIG=false
SUPERSET_PORT=8088
OIDC_ENABLE=True
OIDC_OPENID_REALM=demo

NOTA: I remove my environment variables related to mysql and redis.

Thanks for your help. Tell me if some information is missing.

fedepad commented 2 years ago

Questions which I think are helpful for debugging:

PatBriPerso commented 2 years ago

Thanks @fedepad.

In fact, I change the role of my user with the superset UI and give him the Admin role. Here is how I do that:

I have no specific roles for my keycloak user. I think he has the default keycloak roles but I do not know if those roles are sent to superset. I think roles on keycloak and roles on superset are not related (but I'm not sure).

rahul149386 commented 2 years ago

@fedepad @PatBriPerso I created same user in keycloak and superset eventhough im able to see None user inevent log and througing timeout error any help regarding this. Seeing below

rahul149386 commented 2 years ago

@fedepad @PatBriPerso I created same user in keycloak and superset eventhough im able to see None user inevent log and througing timeout error any help regarding this. Seeing below

![Uploading 20221019_213835.jpg…]()

debimishra89 commented 2 years ago

Can we do role mapping between keycloak and superset with AUTH_TYPE=AUTH_OID?? Or is it only available with AUTH_OAUTH

rusackas commented 9 months ago

@PatBriPerso are you still facing this issue, or should we close it?

resulraveendran commented 7 months ago

Hi,

I got the same issue when i integrate with Zitadel is there any solution for this.

leezj-cbm commented 4 months ago

Having the same issue as OP. Unable to access User List as Admin.

Unable to access User Info as Admin, Alpha and Gamma. Symptoms similar to described as above.

Any advise given will be appreciated.

alexvaut commented 5 days ago

I confirm I have the same issue, if I disable OIDC auth, then I can access user list and user info. If I enable it then I get an "Access Denied". Superset version: 4.1.1 keycloak version: 26.0.5 I use some kind of role mapping between the roles in keycloak and the roles in superset. My logic is implemented in my CUSTOM_SECURITY_MANAGER.

It used to work well with superset version 2.1.0.