search

geosolutions-it / C195-azure-workspace

1 stars 2 forks source link

Add ADFS extension to CKAN #24

Closed etj closed 3 years ago

etj commented 3 years ago

Packages to be installed

The container running CKAN needs:

apt install swig

Install CKAN extension

New CKAN extension to be deployed:

git clone

Then

cd ckanext-azure-auth
pip install -r requirements.txt
pip install -e .

New configuration vars

These values are not needed in the azure ARM, but we may want to add such params in the config json file in order to have a single point of configuration for the admin.

ckan.ini changes

Add plugin azure_auth

Set items:

ckanext.azure_auth.tenant_id = TENANT_ID
ckanext.azure_auth.client_id = CLIENT_ID
ckanext.azure_auth.audience = CLIENT_ID
ckanext.azure_auth.client_secret = CLIENT_SECRET
ckanext.azure_auth.auth_callback_path = /azure/callback
ckanext.azure_auth.allow_create_users = True

Also refer to https://github.com/geosolutions-it/c195-CKAN/issues/1

lpasquali commented 3 years ago

As we spoke lately yesterday @etj with "null values" like:

ENV TENANT_ID 00000000-0000-0000-0000-000000000000
ENV CLIENT_ID 00000000-0000-0000-0000-000000000000
ENV CLIENT_SECRET pKf_0000000_00000-0000-0000

I get this result:

luca@vanth:~/Development/CREA/C195-azure-workspace/ckan-docker$ docker-compose logs -f ckan
Attaching to ckan
ckan          | ckan_db:5432 - accepting connections
ckan          | Generating config at /etc/ckan/production.ini...
ckan          | changed: --set /etc/ckan/production.ini app:main ckan.plugins c195
ckan          | changed: --set /etc/ckan/production.ini app:main ckan.plugins dcat
ckan          | changed: --set /etc/ckan/production.ini app:main ckan.plugins dcat_json_interface
ckan          | changed: --set /etc/ckan/production.ini app:main ckan.plugins structured_data
ckan          | changed: --set /etc/ckan/production.ini app:main ckan.plugins azure_auth
ckan          | changed: --set /etc/ckan/production.ini app:main sqlalchemy.pool_size 10
ckan          | changed: --set /etc/ckan/production.ini app:main sqlalchemy.echo_pool True
ckan          | changed: --set /etc/ckan/production.ini app:main sqlalchemy.pool_pre_ping True
ckan          | changed: --set /etc/ckan/production.ini app:main sqlalchemy.pool_reset_on_return rollback
ckan          | changed: --set /etc/ckan/production.ini app:main sqlalchemy.pool_timeout 30
ckan          | changed: --set /etc/ckan/production.ini DEFAULT debug True
ckan          | changed: --set /etc/ckan/production.ini logger_root level DEBUG
ckan          | changed: --set /etc/ckan/production.ini logger_werkzeug level DEBUG
ckan          | changed: --set /etc/ckan/production.ini logger_ckan level DEBUG
ckan          | unchanged: --set /etc/ckan/production.ini logger_ckanext level DEBUG
ckan          | changed: --set /etc/ckan/production.ini handler_console level DEBUG
ckan          | Setting var and venv...
ckan          | Initting DB...
ckan          | 2021-05-04 08:25:03,812 INFO  [ckan.cli] Using configuration file /etc/ckan/production.ini
ckan          | 2021-05-04 08:25:03,812 INFO  [ckan.config.environment] Loading static files from public
ckan          | 2021-05-04 08:25:03,813 DEBUG [ckan.lib.webassets_tools] Base path /usr/lib/ckan/venv/src/ckan/ckan/public/base
ckan          | 2021-05-04 08:25:04,044 INFO  [ckan.config.environment] Loading templates from /usr/lib/ckan/venv/src/ckan/ckan/templates
ckan          | 2021-05-04 08:25:04,135 DEBUG [ckan.logic] check access OK - get_site_user user=None
ckan          | 2021-05-04 08:25:04,167 INFO  [sqlalchemy.pool.impl.QueuePool] Pool disposed. Pool size: 10  Connections in pool: 0 Current Overflow: -10 Current Checked out connections: 0
ckan          | 2021-05-04 08:25:04,167 INFO sqlalchemy.pool.impl.QueuePool Pool disposed. Pool size: 10  Connections in pool: 0 Current Overflow: -10 Current Checked out connections: 0
ckan          | 2021-05-04 08:25:04,167 INFO sqlalchemy.pool.impl.QueuePool Pool recreating
ckan          | 2021-05-04 08:25:04,167 INFO  [sqlalchemy.pool.impl.QueuePool] Pool recreating
ckan          | 2021-05-04 08:25:04,168 DEBUG [ckan.plugins.core] Loading the synchronous search plugin
ckan          | 2021-05-04 08:25:04,184 INFO  [rdflib] RDFLib Version: 4.2.1
ckan          | 2021-05-04 08:25:04,279 DEBUG [ckan.lib.webassets_tools] Base path /usr/lib/ckan/venv/src/ckan/ckan/public/base
ckan          | Traceback (most recent call last):
ckan          |   File "/usr/lib/ckan/venv/bin/ckan", line 33, in <module>
ckan          |     sys.exit(load_entry_point('ckan', 'console_scripts', 'ckan')())
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
ckan          |     return self.main(*args, **kwargs)
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 781, in main
ckan          |     with self.make_context(prog_name, args, **extra) as ctx:
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 700, in make_context
ckan          |     self.parse_args(ctx, args)
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 1212, in parse_args
ckan          |     rest = Command.parse_args(self, ctx, args)
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 1048, in parse_args
ckan          |     value, args = param.handle_parse_result(ctx, opts, args)
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 1630, in handle_parse_result
ckan          |     value = invoke_param_callback(self.callback, ctx, self, value)
ckan          |   File "/usr/lib/ckan/venv/lib/python3.7/site-packages/click/core.py", line 123, in invoke_param_callback
ckan          |     return callback(ctx, param, value)
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/cli/cli.py", line 102, in _init_ckan_config
ckan          |     ctx.obj = CkanCommand(value)
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/cli/cli.py", line 52, in __init__
ckan          |     self.app = make_app(self.config)
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/config/middleware/__init__.py", line 56, in make_app
ckan          |     load_environment(conf)
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/config/environment.py", line 122, in load_environment
ckan          |     p.load_all()
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/plugins/core.py", line 155, in load_all
ckan          |     load(*plugins)
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/plugins/core.py", line 183, in load
ckan          |     plugins_update()
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/plugins/core.py", line 137, in plugins_update
ckan          |     environment.update_config()
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/config/environment.py", line 176, in update_config
ckan          |     plugin.update_config(config)
ckan          |   File "/usr/lib/ckan/venv/src/ckanext-azure-auth/ckanext/azure_auth/plugin.py", line 58, in update_config
ckan          |     if not config[ATTR_TENANT_ID] and not config[ATTR_AD_SERVER]:
ckan          |   File "/usr/lib/ckan/venv/src/ckan/ckan/common.py", line 98, in __getitem__
ckan          |     return self.store[key]
ckan          | KeyError: 'ckanext.azure_auth.tenant_id'
ckan exited with code 1

looks like it is not liking dummy values as I took from https://github.com/geosolutions-it/c195-CKAN/issues/1 as a default so I put a check in the entrypoint as well in evironment to disable it at need, because without it it might break local development of C195-azure-workspace, so by default plugin is disabled (but built and present in ckan image) or enabled by putting to true/false CKAN_AZURE_AUTH_ENABLED (since you told me this has to be momentary as it is not the expected behavior I did not update the ckan-doker readme about this). work done so far is here, let me know if I should go on testing (on azure environment, I tested locally env generation and docker image but before PR I should try a deploy on azure) and create PR as is or should we wait for plugin fix and then remove the enable/disable check in the entrypoint and its variable. https://github.com/geosolutions-it/C195-azure-workspace/tree/issue_24_ADFS_ckan_plugin_addition

etj commented 3 years ago

Latest commit makes it possible for CKAN to run even if there is some misconfiguration that prevents CKAN to request info from the azure ADFS.

@lpasquali can you pls test it in the docker architecture?

lpasquali commented 3 years ago

@etj 

2021-05-06 16:18:56,297 DEBUG [ckanext.azure_auth.auth_config] Loading ADFS ID Provider configuration.
2021-05-06 16:18:56,298 INFO  [ckanext.azure_auth.auth_config] Trying to get OpenID Connect config from https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/.well-known/openid-configuration?appid=00000000-0000-0000-0000-000000000000
2021-05-06 16:18:56,584 INFO  [ckanext.azure_auth.auth_config] Trying to get ADFS Metadata file https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/FederationMetadata/2007-06/FederationMetadata.xml
2021-05-06 16:18:56,743 CRITI [ckanext.azure_auth.auth_config] Could not load any data from ADFS server. Authentication against ADFS is not possible. 
2021-05-06 16:18:56,743 CRITI [ckanext.azure_auth.plugin] Could not load any data from ADFS server. Authentication against ADFS is not possible. 
2021-05-06 16:18:56,745 INFO  [ckan.config.environment] Loading templates from /usr/lib/ckan/venv/src/ckan/ckan/templates
2021-05-06 16:18:56,748 DEBUG [ckan.logic] check access OK - get_site_user user=None
[...]
2021-05-06 16:32:28,026 DEBUG [ckan.lib.redis] Using Redis at redis://ckan_redis:6379/1
2021-05-06 16:32:28,040 DEBUG [ckan.lib.app_globals] config `ckan.site_title` set to `CKAN` from config
2021-05-06 16:32:28,048 DEBUG [ckan.lib.app_globals] config `ckan.site_logo` set to `/base/images/ckan-logo.png` from config
2021-05-06 16:32:28,052 DEBUG [ckan.lib.app_globals] config `ckan.site_url` set to `http://localhost:5000` from config
2021-05-06 16:32:28,067 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.metadata_url` set to `https://login.microsoftonline.com/` from config
2021-05-06 16:32:28,069 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.redirect_uri` set to `http://localhost:5000/azure/callback` from config
2021-05-06 16:32:28,070 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.tenant_id` set to `00000000-0000-0000-0000-000000000000` from config
2021-05-06 16:32:28,071 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.client_id` set to `00000000-0000-0000-0000-000000000000` from config
2021-05-06 16:32:28,072 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.client_secret` set to `pKf_0000000_00000-0000-0000` from config
2021-05-06 16:32:28,075 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.ad_server` set to `https://login.microsoftonline.com` from config
2021-05-06 16:32:28,076 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.allow_create_users` set to `True` from config
2021-05-06 16:32:28,077 DEBUG [ckan.lib.app_globals] config `ckanext.azure_auth.audience` set to `00000000-0000-0000-0000-000000000000` from config

made PR: https://github.com/geosolutions-it/C195-azure-workspace/pull/34

etj commented 3 years ago

In the deployed container I see:

root@5d6a139fe546:/# grep azure_auth /etc/ckan/production.ini    
ckan.plugins = stats text_view image_view recline_view c195 dcat dcat_json_interface structured_data azure_auth grace_period
ckanext.azure_auth.tenant_id = 00000000-0000-0000-0000-000000000000
ckanext.azure_auth.client_id = 00000000-0000-0000-0000-000000000000
ckanext.azure_auth.audience = 00000000-0000-0000-0000-000000000000
ckanext.azure_auth.client_secret = pKf_0000000_00000-0000-0000
ckanext.azure_auth.auth_callback_path = /azure/callback
ckanext.azure_auth.allow_create_users = True

even if I put the proper values in parameters.json

etj commented 3 years ago

It seems that also configuring the requested params in the json file, the ckan ini file is not properly modified.

Are these lines supposed to be set that way @lpasquali ?

https://github.com/geosolutions-it/C195-azure-workspace/blob/8ddc364ff39da9677d80678258aaccd9d6f59f25/ckan-docker/Dockerfile#L40-L42

lpasquali commented 3 years ago

It seems that also configuring the requested params in the json file, the ckan ini file is not properly modified.

Are these lines supposed to be set that way @lpasquali ?

https://github.com/geosolutions-it/C195-azure-workspace/blob/8ddc364ff39da9677d80678258aaccd9d6f59f25/ckan-docker/Dockerfile#L40-L42 actually the three vairables were just missing in the compose definitios, I added them, adding also a more meaningful prefix to recognise theri fucntion outside the docker image. https://github.com/geosolutions-it/C195-azure-workspace/pull/38 @etj