Persistent Brick index for Self Heal xattr names

[aravindavk]

Afr records dirty flag details in xattr. The name of the xattr is <volume-name>-client-<index>

These xattrs names are specified as afr-pending-xattr in volfile so that both Client and Self heal daemon will understand. For example,

volume testvol-replicate-0
    type cluster/replicate
    option iam-self-heal-daemon yes
    option use-compound-fops off
    option self-heal-daemon enable
    option entry-self-heal on
    option data-self-heal on
    option metadata-self-heal on
    option background-self-heal-count 0
    option afr-pending-xattr testvol-client-0,testvol-client-1,testvol-client-2
    subvolumes testvol-client-0 testvol-client-1 testvol-client-2
end-volume

The index starts from zero and will not reset for each sub volume. If a brick is removed, index will not change for existing bricks. When new bricks added it will get new index as len(volinfo.Bricks) + 1

As a alternative, Brick ID can be used as part of xattr name <volume-name>-client-<brick-id>. But this will break the backward compatibility.

This patch in Gd1 introduces persistent brick index.

[aravindavk]

@itisravi please add if I missed anything

[amarts]

I would like to keep the subvol names as brick id, but the hard part is it gets messed up with debugging. Another way to solve the exact upgrade issue is, to handle the subvol ordering (naming) in volume create itself in the json request. That way, if the names are provided, volgen would use it, if not, it will assign brick-id ?

[aravindavk]

I would like to keep the subvol names as brick id, but the hard part is it gets messed up with debugging. Another way to solve the exact upgrade issue is, to handle the subvol ordering (naming) in volume create itself in the json request. That way, if the names are provided, volgen would use it, if not, it will assign brick-id ?

The issue is not related to volgen, issue related to heal xattr names. Problem exists for xattrs within subvolume too.

• If a brick is replaced then it should not get same index, because if xattr name belongs to existing brick. Heal will not happen to the new brick
• Remove brick and add brick should not get same index

[itisravi]

If a brick is replaced then it should not get same index

actually, during replace brick , the new brick does get the same index as the old one.

[itisravi]

As a alternative, Brick ID can be used as part of xattr name

Is this Brick ID different from the one in GD1? In GD1, AFR uses this BrickID only (introduced in the patch you referred to in the description above) as the xattr name: https://review.gluster.org/#/c/12738/

[aravindavk]

Is this Brick ID different from the one in GD1? In GD1, AFR uses this BrickID only (introduced in the patch you referred to in the description above) as the xattr name: https://review.gluster.org/#/c/12738/

Brick ID in Gd2 is not sequence number but it is like UUID.

[aravindavk]

To summarize,

• On Volume create start brick index from 0.
• Continue the numbering across the sub volumes.
• On Replace brick, reuse the same index as previous brick
• Add brick always gets max_index+1
• Index can be reused if complete sub volume is removed and new sub volume is added.

[aravindavk]

Removing GCS-Blocker label from this issue since this issue is not applicable with single sub volume.