sscargal / pmemchk

MIT License
0 stars 1 forks source link

[Optane] Check ndctl 'numa_node' matches the physical NUMA node ID #116

Open sscargal opened 2 years ago

sscargal commented 2 years ago

The 'numa_node' reported by ndctl list -N should match the physical NUMA node

# ndctl list -N -v
[
  {
    "dev":"namespace1.0",
    "mode":"fsdax",
    "map":"dev",
    "size":1065418227712,
    "uuid":"8420bbe0-31de-4df8-87f0-94148d7121ee",
    "raw_uuid":"70431086-7b4c-4e01-9a0d-80baad90ecd7",
    "sector_size":512,
    "align":2097152,
    "blockdev":"pmem1",
    "numa_node":0,  <<<<< Should be 1
    "target_node":3
  },
  {
    "dev":"namespace0.0",
    "mode":"fsdax",
    "map":"dev",
    "size":1065418227712,
    "uuid":"dcfe052c-258e-40f9-9e79-625dccb4b2a6",
    "raw_uuid":"9dbdd382-3bcd-43b9-8a17-3c51743cc6c0",
    "sector_size":512,
    "align":2097152,
    "blockdev":"pmem0",
    "numa_node":0, <<< Good
    "target_node":2
  }
]

A simple test is to check the numa_node value matches the dev value. eg: (numa_node = 1) == (namespace1.0), where namespaceX.Y -> X = NUMA Node, Y = Instance.

This could also be cross-referenced with the 'numa_node' values in /sys

# cat /sys/bus/nd/devices/namespace*/numa_node
0
0
0 <<< Should be 1
0 <<< Should be 1

Two physical NUMA nodes exist

# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77
node 0 size: 128334 MB
node 0 free: 122292 MB
node 1 cpus: 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103
node 1 size: 129010 MB
node 1 free: 119254 MB
node distances:
node 0 1
0: 10 20
1: 20 10

Need to be aware of Sub-NUMA Clustering