UtkarshBhatthere commented 3 weeks ago

Description

Add RGW Keystone configurations to MicroCeph cluster config intterface

Fixes #360

Type of change

Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] CleanCode (Code refactor, test updates, does not introduce functional changes)
[ ] Documentation update (Doc only change)

How Has This Been Tested?

T.B.A

Contributor's Checklist

Please check that you have:

[ ] self-reviewed the code in this PR.
[ ] added code comments, particularly in hard-to-understand areas.
[ ] updated the user documentation with corresponding changes.
[ ] added tests to verify effectiveness of this change.

hemanthnakkina commented 3 weeks ago

@UtkarshBhatthere Thanks for the PR. Can you add couple of more configs.

hemanthnakkina commented 3 weeks ago

Tested swift by setting the following options.

sudo microceph cluster config set rgw_keystone_url http://10.20.21.12/openstack-keystone/
sudo microceph cluster config set rgw_keystone_admin_user svc_remote_70e68fbe4b73410685e59e20bdb33add-IvU7VA
sudo microceph cluster config set rgw_keystone_admin_password REDACTED
sudo microceph cluster config set rgw_keystone_api_version 3
sudo microceph cluster config set rgw_keystone_admin_domain service_domain
sudo microceph cluster config set rgw_keystone_admin_project services
sudo microceph cluster config set rgw_keystone_accepted_roles Member,member
sudo microceph cluster config set rgw_keystone_accepted_admin_roles Admin
sudo microceph cluster config set rgw_keystone_token_cache_size 500
sudo microceph cluster config set rgw_keystone_verify_ssl false
sudo microceph cluster config set rgw_keystone_service_token_enabled true
sudo microceph cluster config set rgw_keystone_service_token_accepted_roles Admin

Created container using openstack commands and it is successful.

$ openstack container create foo
+---------+-----------+-------------------------------------------------+
| account | container | x-trans-id                                      |
+---------+-----------+-------------------------------------------------+
| v1      | foo       | tx0000052e64bfb4f7504d4-006667beeb-8691-default |
+---------+-----------+-------------------------------------------------+

$ openstack container list +------+ | Name | +------+ | foo | +------+

Verified manually by adding S3 config rgw s3 auth use keystone = true to /var/snap/microceph/current/conf/radosgw.conf test_s3.py - https://pastebin.canonical.com/p/YStnRKNYt3/

$ openstack ec2 credentials create +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+ | access | REDACTED | | links | {'self': 'http://10.20.21.11/openstack-keystone/v3/users/1044dd9ded5b401597e8dfa29123a8d1/credentials/OS-EC2/19c1587bc0b242bc817154bbb13c1233'} | | project_id | baf39e32eb7a4bb284833c3879a4cf0a | | secret | REDACTED | | trust_id | None | | user_id | 1044dd9ded5b401597e8dfa29123a8d1 | +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+

$ python3 test_s3.py foo 2024-06-11 03:05:15.400000+00:00

hemanthnakkina commented 3 weeks ago

@UtkarshBhatthere One issue with this PR.

Tried to set rgw related configs when the service is disabled, microceph tries to start rgw service and fails. microceph should not try to start the service as the service is disabled.

Jun 11 09:10:41 sunbeam1 systemd[1]: Started Service for snap application microceph.rgw.
Jun 11 09:10:41 sunbeam1 microceph.rgw[1022474]: global_init: unable to open config file from search list /var/snap/microceph/x1/conf/radosgw.conf
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Failed with result 'exit-code'.
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Scheduled restart job, restart counter is at 1.
Jun 11 09:10:41 sunbeam1 systemd[1]: Stopped Service for snap application microceph.rgw.
Jun 11 09:10:41 sunbeam1 systemd[1]: Started Service for snap application microceph.rgw.
Jun 11 09:10:41 sunbeam1 microceph.rgw[1022508]: global_init: unable to open config file from search list /var/snap/microceph/x1/conf/radosgw.conf
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Failed with result 'exit-code'.
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Scheduled restart job, restart counter is at 2.
Jun 11 09:10:41 sunbeam1 systemd[1]: Stopped Service for snap application microceph.rgw.
Jun 11 09:10:41 sunbeam1 systemd[1]: Started Service for snap application microceph.rgw.
Jun 11 09:10:41 sunbeam1 microceph.rgw[1022535]: global_init: unable to open config file from search list /var/snap/microceph/x1/conf/radosgw.conf
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 09:10:41 sunbeam1 systemd[1]: snap.microceph.rgw.service: Failed with result 'exit-code'.
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Scheduled restart job, restart counter is at 3.
Jun 11 09:10:42 sunbeam1 systemd[1]: Stopped Service for snap application microceph.rgw.
Jun 11 09:10:42 sunbeam1 systemd[1]: Started Service for snap application microceph.rgw.
Jun 11 09:10:42 sunbeam1 microceph.rgw[1022582]: global_init: unable to open config file from search list /var/snap/microceph/x1/conf/radosgw.conf
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Failed with result 'exit-code'.
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Scheduled restart job, restart counter is at 4.
Jun 11 09:10:42 sunbeam1 systemd[1]: Stopped Service for snap application microceph.rgw.
Jun 11 09:10:42 sunbeam1 systemd[1]: Started Service for snap application microceph.rgw.
Jun 11 09:10:42 sunbeam1 microceph.rgw[1022609]: global_init: unable to open config file from search list /var/snap/microceph/x1/conf/radosgw.conf
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Failed with result 'exit-code'.
Jun 11 09:10:42 sunbeam1 systemd[1]: snap.microceph.rgw.service: Scheduled restart job, restart counter is at 5.

UtkarshBhatthere commented 3 weeks ago

added a service age check for rgw, it now stays up post bulk config change: @hemanthnakkina

$ sudo systemctl status snap.microceph.rgw
● snap.microceph.rgw.service - Service for snap application microceph.rgw
     Loaded: loaded (/etc/systemd/system/snap.microceph.rgw.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-06-14 16:37:12 IST; 1min 4s ago
   Main PID: 234740 (radosgw)
      Tasks: 53 (limit: 18198)
     Memory: 14.8M
        CPU: 108ms
     CGroup: /system.slice/snap.microceph.rgw.service
             └─234740 radosgw -f --cluster ceph --name client.radosgw.gateway -c /var/snap/microceph/x1/conf/radosgw.conf

Jun 14 16:37:12 workbook systemd[1]: Started Service for snap application microceph.rgw.

hemanthnakkina commented 3 weeks ago

@UtkarshBhatthere thanks for the PR! Can you update Contributors Checklist etc

canonical / microceph

Added RGW keystone integration configurations #363

Description

Type of change

How Has This Been Tested?

Contributor's Checklist