Ceph CRUD Flows

[shtripat]

At the moment Calamri provides crud operations like create/edit/delete for storage pools only. For other entities like RBDs, erasure coded pools etc we need to consume CLI endpoint from Calamari. The point I have is, do we really need Calamari (another daemon) running on MON nodes only for this purpose? Cant we look at using rados command execution directly from ceph-integration component itselt?

@Tendrl/tendrl-core suggestions/comments?

[shtripat]

@GregMeno I have tried running Calamari on each monitor node (in a cluster with 3 mons and 3 osd nodes) and if I try to create a pool from one calamari session, I am able to see it reflected for other calamari properly. Now the question I have is whether this model has some concerns or its OK. (I remember earlier intentionally we were running Calamari on only one mon node as they have own instances of DB running and there could be a mismatch between the data they are aware of). Is it fine if we use Calamari for posting CRUD operations to the ceph cluster with this model?

[brainfunked]

@shtripat would you mind investigating the rados command availability for all the crud operations we're looking to implement and document them here?

[shtripat]

The list of valid operations entity wise are -

1. Pool - delete/update/create
2. OSD - update/scrub/depp-scrub/update_config/repair
3. CRUSH - update
4. CRUSH Node - update/create/delete/add_bucket/remove_bucket/move_bucket/reweight_osd/move_osd

In addition to these we can run rados_command and rbd_command using utility available with https://github.com/Tendrl/ceph-integration/blob/master/tendrl/ceph_integration/ceph.py

@r0h4n correct if anything wrong or missed out here.

[r0h4n]

@shtripat Yes, these are the capabilities http://docs.ceph.com/docs/giant/rados/api/python/

in addition to what you mentioned

[shtripat]

Looks everything we need at the moment is achievable using rados and rbd command mechanism (if I am correct)

[mbukatov]

Looks everything we need at the moment is achievable using rados and rbd command mechanism (if I am correct)

That implies that Tendrl needs to be able to deploy ceph auth. keys (during cluster creation or import) for itself to be able to work with a ceph cluster. Do I understand this right?

[ChristinaMeno]

@shtripat It's fine as long as you're not randomly doing each operation and pin requests and responses. I would recommend something like https://github.com/ceph/calamari/blob/master/cthulhu/cthulhu/manager/cluster_monitor.py#L258 Where you choose a calamari to talk to and rotate if it becomes unreliable.

cheers, G