doubt for dht_selfheal_layout_new_directory and trusted.glusterfs.mdata

[perrynzhou]

@csabahenk @mykaul @mohit84 @itisravi @amarts @hari when first mount volume taht will call dht_selfheal_layout_new_directory,but when second mount with different mount dir, dht_selfheal_layout_new_directory not be called,cal you tell me why?

[2020-08-31 01:29:57.098107] T [MSGID: 0] [client-rpc-fops_v2.c:2633:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7fdf040011e8, dht3-client-0 returned 0
[2020-08-31 01:29:57.098139] D [MSGID: 0] [dht-common.c:1382:dht_lookup_dir_cbk] 0-dht3-dht: /: lookup on dht3-client-0 returned with op_ret = 0, op_errno = 0
[2020-08-31 01:29:57.098149] T [MSGID: 0] [dht-layout.c:347:dht_layout_merge] 0-dht3-dht: Missing disk layout on dht3-client-0. err = -1
[2020-08-31 01:29:57.098162] D [MSGID: 0] [dht-common.c:1478:dht_lookup_dir_cbk] 0-dht3-dht: /: mds xattr trusted.glusterfs.dht.mds is not present on dht3-client-0(gfid =
 00000000-0000-0000-0000-000000000001)
[2020-08-31 01:29:57.098180] I [MSGID: 109063] [dht-layout.c:641:dht_layout_normalize] 0-dht3-dht: Found anomalies [{path=/}, {gfid=00000000-0000-0000-0000-000000000001},
 {holes=1}, {overlaps=0}]
[2020-08-31 01:29:57.098187] D [MSGID: 0] [dht-common.c:1325:dht_needs_selfheal] 0-dht3-dht: fixing assignment on /
[2020-08-31 01:29:57.098207] D [MSGID: 0] [dht-selfheal.c:1763:dht_selfheal_layout_new_directory] 0-dht3-dht: chunk size = 0xffffffff / 41949 = 102385.451262
[2020-08-31 01:29:57.098214] T [MSGID: 0] [dht-hashfn.c:95:dht_hash_compute] 0-dht3-dht: trying regex for /
[2020-08-31 01:29:57.098228] D [MSGID: 0] [dht-selfheal.c:1799:dht_selfheal_layout_new_directory] 0-dht3-dht: assigning range size 0x55555555 to dht3-client-1
[2020-08-31 01:29:57.098234] T [MSGID: 0] [dht-selfheal.c:1800:dht_selfheal_layout_new_directory] 0-dht3-dht: gave fix: 0x0 - 0x55555554, with commit-hash 0x1 on dht3-cli
ent-1 for /
[2020-08-31 01:29:57.098240] D [MSGID: 0] [dht-selfheal.c:1799:dht_selfheal_layout_new_directory] 0-dht3-dht: assigning range size 0x55555555 to dht3-client-2
[2020-08-31 01:29:57.098245] T [MSGID: 0] [dht-selfheal.c:1800:dht_selfheal_layout_new_directory] 0-dht3-dht: gave fix: 0x55555555 - 0xaaaaaaa9, with commit-hash 0x1 on d
ht3-client-2 for /
[2020-08-31 01:29:57.098251] D [MSGID: 0] [dht-selfheal.c:1799:dht_selfheal_layout_new_directory] 0-dht3-dht: assigning range size 0x55555555 to dht3-client-0
[2020-08-31 01:29:57.098256] T [MSGID: 0] [dht-selfheal.c:1800:dht_selfheal_layout_new_directory] 0-dht3-dht: gave fix: 0xaaaaaaaa - 0xfffffffe, with commit-hash 0x1 on d
ht3-client-0 for /

@csabahenk @mykaul @mohit84 @itisravi @amarts @hari what's meaning for trusted.glusterfs.mdata and trusted.glusterfs.mdata?is has way to trace with gdb for that attribute?

[root@CentOS1 ~]$ getfattr  -d -m . -e hex /glusterfs/dht/brick1 
getfattr: Removing leading '/' from absolute path names
# file: glusterfs/dht/brick1
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a64656661756c745f743a733000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x0000000100000000aaaaaaaaffffffff
trusted.glusterfs.mdata=0x010000000000000000000000005f4c52950000000006421021000000005f4c51ec000000000c071c00000000005f4c5295000000000272bfd1
trusted.glusterfs.volume-id=0xafe16957a35147a09a5934c23ba0e09a

[root@CentOS1 ~]$ getfattr  -d -m . -e hex /glusterfs/dht/brick2
getfattr: Removing leading '/' from absolute path names
# file: glusterfs/dht/brick2
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a64656661756c745f743a733000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x00000001000000000000000055555554
trusted.glusterfs.mdata=0x010000000000000000000000005f4c52950000000006421021000000005f4c51ec000000000c071c00000000005f4c5295000000000272bfd1
trusted.glusterfs.volume-id=0xafe16957a35147a09a5934c23ba0e09a

[root@CentOS1 ~]$ getfattr  -d -m . -e hex /glusterfs/dht/brick3
getfattr: Removing leading '/' from absolute path names
# file: glusterfs/dht/brick3
security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a64656661756c745f743a733000
trusted.gfid=0x00000000000000000000000000000001
trusted.glusterfs.dht=0x000000010000000055555555aaaaaaa9
trusted.glusterfs.mdata=0x010000000000000000000000005f4c52950000000006421021000000005f4c51ec000000000c071c00000000005f4c5295000000000272bfd1
trusted.glusterfs.volume-id=0xafe16957a35147a09a5934c23ba0e09a

[mohit84]

The dht_selfheal_layout_new_directory call's at the time of set layout on a directory at backend. As you can see logs are throwing below messages

[2020-08-31 01:29:57.098180] I [MSGID: 109063] [dht-layout.c:641:dht_layout_normalize] 0-dht3-dht: Found anomalies [{path=/}, {gfid=00000000-0000-0000-0000-000000000001}, {holes=1}, {overlaps=0}] [2020-08-31 01:29:57.098187] D [MSGID: 0] [dht-common.c:1325:dht_needs_selfheal] 0-dht3-dht: fixing assignment on /

Once layout has been configured and volume has been mount on another client dht_lookup_dir_cbk has found the layout and it does not call the function dht_selfheal_layout_new_directory.There are other condition also when dht call's this function like when dht found the xattr needs to be heal on backend dht call's this function.

The xattr ( trusted.glusterfs.mdata) is specific to ctime features if ctime feature is enable the xattr has been populate on the backend. By default feature is enabled that's why you are seeing this parameter.

[perrynzhou]

The dht_selfheal_layout_new_directory call's at the time of set layout on a directory at backend. As you can see logs are throwing below messages

[2020-08-31 01:29:57.098180] I [MSGID: 109063] [dht-layout.c:641:dht_layout_normalize] 0-dht3-dht: Found anomalies [{path=/}, {gfid=00000000-0000-0000-0000-000000000001}, {holes=1}, {overlaps=0}] [2020-08-31 01:29:57.098187] D [MSGID: 0] [dht-common.c:1325:dht_needs_selfheal] 0-dht3-dht: fixing assignment on /

Once layout has been configured and volume has been mount on another client dht_lookup_dir_cbk has found the layout and it does not call the function dht_selfheal_layout_new_directory.There are other condition also when dht call's this function like when dht found the xattr needs to be heal on backend dht call's this function.

The xattr ( trusted.glusterfs.mdata) is specific to ctime features if ctime feature is enable the xattr has been populate on the backend. By default feature is enabled that's why you are seeing this parameter.

@mohit84 thank you very much.i got it