search

rcbops / rpc-ceph

Other
2 stars 14 forks source link

readme

Decom Note:

rpc-ceph is no longer being developed or tested. Please use the upstream ceph-ansible playbooks for any future deployments.

rpc-ceph and ceph-ansible

rpc-ceph deploys Ceph as an RPC stand-alone platform in a uniform, managed, and tested way to ensure version consistency and testing.

By adding automated tests, rpc-ceph provides a way to manage tested versions of ceph-ansible used in RPC deployments.

Current versions of ceph-ansible & Ansible

ceph-ansible version: v3.1.12

Ansible version: 2.4.4.0

What is rpc-ceph?

rpc-ceph is a thin wrapper around the ceph-ansible project. rpc-ceph manages the versions of ansible and ceph-ansible by providing:

Deploying rpc-ceph uses boostrap.sh, ceph-ansible, default group_vars, and a pre-created playbook.

NOTE: Anything that can be configured with ceph-ansible is configurable with rpc-ceph.

Deploying Ceph for multi-node and production environments

Architecture

We do not recommend or use containers for rpc-ceph production deployments. Containers are setup and used as part of the run_tests.sh (AIO) testing strategy only. The default playbooks are not set up to build containers or configure any of the required container specific roles.

The inventory should consist of the following:

Configuring your deployment host

  1. Configure the following inventory:

    • ansible_host var for each host.
    • Devices, dedicated_devices for osd hosts.

  2. Configure a variables file including the following ceph-ansible vars:

    • monitor_interface
    • public_network
    • cluster_network
    • osd_scenario
    • repo_server_interface
    • Any other ceph-ansible settings you want to configure.

  3. Set any override vars in playbooks/group_vars/host_group/overrides.yml, this allows:

    • Defaults to remain, but be overriden if required (overrides.yml will take precedence).
    • Git will ignore the overrides.yml file, so the repo can be updated without clearing out all deploy specific vars.

  4. Override any variables from ceph.conf using ceph_conf_overrides_extra or ceph_conf_overrides_<group>_extra:

    • This allows the default group_vars to remain in place, and means you do not have to respecify any vars you aren't setting.
    • The ceph_conf_overrides_<group>_extra var will override only vars for only the hosts in that group, with currently supported groups:
      • ceph_conf_overrides_rgw_extra
      • ceph_conf_overrides_mon_extra
      • ceph_conf_overrides_mgr_extra
      • ceph_conf_overrides_osd_extra
    • The overrides will merge with the existing settings and take precedence but not squash them.

  5. Run the bootstrap-ansible.sh inside the scripts directory:

    ./scripts/bootstrap-ansible.sh

  6. This configures ansible at a pre-tested version, creates a ceph-ansible binary that points to the appropriate ansible-playbook binary, and clones the required role repositories:

    • ceph-ansible
    • rsyslog_client
    • openstack-ansible-plugins (ceph-ansible uses the config template plugin from here).
    • haproxy_server
    • rsyslog_server

  7. Run the ceph-ansible playbook from the playbooks directory:

    ceph-ansible-playbook -i <link to your inventory file> playbooks/add-repo.yml -e @<link to your vars file>
ceph-ansible-playbook -i <link to your inventory file> playbooks/deploy-ceph.yml -e @<link to your vars file>

  8. Run any additional playbooks from the playbook directory:

    • ceph-setup-logging.ymlwill setup rsyslog client, ensure you have the appropriate rsyslog server setup, or other log shipping location, refer to: https://docs.openstack.org/openstack-ansible-rsyslog_client/latest/ for more details
    • ceph-keystone-rgw.yml will setup required keystone users and endpoints for Ceph.
    • ceph-rgw-haproxy.yml will setup the HAProxy VIP for Ceph Rados GW. Ensure you specify haproxy_all group in your inventory with the HAProxy hosts.
    • ceph-rsyslog-server.yml will setup rsyslog server on the rsyslog_all hosts specified. NB If there is already an existing rsyslog server that you are connecting into, you should not run this.

Your deployment should be successful.

NOTE: If there are any errors, troubleshoot as a standard ceph-ansible deployment.

Deploying Ceph as an AIO

Virtual Machine requirements for AIO

To run an AIO build

For MaaS integration, perform the following export commands. Otherwise just use ./run_tests.sh to build the AIO.

export PUBCLOUD_USERNAME=<username>
export PUBCLOUD_API_KEY=<api_key>

AIO Scenarios

To run an AIO scenario for Ceph you can run the following export on a general1-8 or perf2-15 flavor instance, unless otherwise noted:

export RE_JOB_SCENARIO="name of scenario from below"

build_releasenotes This will build the project releae notes using sphinx and place it in the directory rpc-ceph/release/build/

functional: This is a base AIO for Ceph, includes MaaS testing, this runs on each commit, with the following components:

This job does not run the benchmarking playbooks.

bluestore: This is the same as the functional job but runs using bluestore, and 3 collocated OSD devices per osd host.

rpco_newton: An RPC-O newton integration test, that will deploy an RPC-O AIO, and integrate it with Ceph, followed by Tempest tests. This runs daily, as it takes a long time to build.

NB: This requires a perf2-15 instance.

rpco_pike This is the same as the rpco_newton job but built against the pike branch of RPC-O.

rpco_queens This is the same as the rpco_newton and rpco-pike jobs but built against the queens branch of RPC-O.

rpco_rocky This is the same as the rpco_newton and rpco-pike jobs but built against the rocky branch of RPC-O.

keystone_rgw: A basic keystone integration test, that will run on each commit. Utilizing the swift client to ensure Keystone integration is working.

Additionally this test runs the FIO and RGW benchmarking playbooks to ensure they work, but does not run the MaaS playbooks.

Currently not supported for AIO