radosgw doesn't work

[prometheanfire]

the ceph.conf file is not populated with the vars needed to talk to keystone

[client.rgw.123456-storage01]
log file = /var/log/ceph/ceph-rgw-123456-storage01.log
host = 123456-storage01
keyring = /var/lib/ceph/radosgw/ceph-rgw.123456-storage01/keyring
rgw frontends = civetweb port=1.2.3.4:8080 num_threads=1024

[prometheanfire]

there is also variable reuse with the radosgw_keystone variable (it controls running ceph-keystone-rgw.yml AND ceph_conf_overrides_rgw (in the group vars).

[andymcc]

Can you elaborate more on that? What was your config? The radosgw_keystone var is designed to enable the integration of radosgw with keystone:

• The required vars for radosgw (hence the ceph_conf_overrides_rgw - this ensures that only radosgw hosts get those vars)
• The playbook required to setup the endpoints.

E.g. if you set it, it will use the default var overrides for radosgw to integrate with keystone (hence the ceph_conf_pverrides_rgw). You still need to set some other vars from the RPC-O/OpenStack side.

There is a gate test, here is a sample of the vars you would need to ensure are set: https://github.com/rcbops/rpc-ceph/blob/master/tests/test-vars-rgw.yml

Of course, not all of them are required, depending on how you setup haproxy etc.

[prometheanfire]

The radosgw config was in the first comment.

What if I want to set up radosgw without setting up the endpoints (since that's not handled by ceph, but by rpc-o). How can I do one and not the other.

I did also notice the docs needed radosgw_keystone_ssl: false set (it doesn't default to true OR false).

So, since we are not running the endpoint-setup playbook (commented out the include until that's fixed). What vars are needed? my vars.yml has the following rgw values.

# RADOS vars
radosgw_civetweb_num_threads: 1024
internal_lb_vip_address: 1.2.3.4
service_region: RegionOne
radosgw_keystone_service_name: swift
radosgw_keystone_service_description: "Swift Service"
radosgw_keystone_admin_user: swift
radosgw_keystone_admin_password: hunter2
radosgw_keystone: true
radosgw_keystone_ssl: false

since I'm not running the endpoint registration the following vars were not set.

keystone_admin_user_name: admin
keystone_auth_admin_password: "SuperSecretePassword"
keystone_admin_tenant_name: admin
keystone_service_adminuri_insecure: false

[prometheanfire]

the openstack side is all set up, we are getting 401s from RGW though (because it looks like it's not set up to use a keystone backend)

[andymcc]

I'll add radosgw_keystone_ssl as a default to false - it should be defaulted in ceph-ansible, but there's an issue for that: https://github.com/ceph/ceph-ansible/issues/2111 - so i'll add a fix upstream (https://github.com/ceph/ceph-ansible/pull/2355).

It should always be false unless you are using PKI keys with keystone.

There will be multple rgw.client stections (one for each rgw host) on each server, check the section related to the host you are looking at, e.g. on host storage01, it should include more details in the storage01 section, but will include minimal settings for the other hosts e.g. on host rgw1:

[client.rgw.allsvc] log file = /var/log/ceph/ceph-rgw-allsvc.log host = allsvc keyring = /var/lib/ceph/radosgw/ceph-rgw.allsvc/keyring rgw frontends = civetweb port=10.1.1.103:8080 num_threads=4096

[client.rgw.rgw1] rgw_s3_auth_use_keystone = True rgw_keystone_admin_password = testpass rgw_keystone_url = http://10.1.1.1:35357 rgw_keystone_accepted_roles = Member, member, admin rgw_keystone_admin_tenant = service keyring = /var/lib/ceph/radosgw/ceph-rgw.rgw1/keyring host = rgw1 rgw_keystone_api_version = 3 rgw_keystone_admin_user = radosgw rgw_keystone_admin_domain = default log file = /var/log/ceph/ceph-rgw-rgw1.log rgw frontends = civetweb port=10.1.1.106:8080 num_threads=4096

401 suggests it knows where to connect but can't auth, not that the endpoint is unavailable.

[prometheanfire]

This is a single host deploy, so only one section. The section I pasted in comment one was the only rgw config in ceph.conf.

I wonder if it's not happy the inventory used ip addresses and not hostnames.

---
all:
  hosts:
    1.2.3.4:
      dedicated_devices:
        - /dev/sde
        - /dev/sdf
  children:
    rgws:
      hosts:
        1.2.3.4

[prometheanfire]

changed to the following with the same results :|

---
all:
  hosts:
    123456-storage01:
      dedicated_devices:
        - /dev/sde
        - /dev/sdf
      ip: 1.2.3.4
  children:
    rgws:
      hosts:
        123456-storage01

[andymcc]

Can we close this one out @prometheanfire ?

[prometheanfire]

yep