search

openstreetmap / operations

OSMF Operations Working Group issue tracking
https://operations.osmfoundation.org/
99 stars 12 forks source link

Resize NVMe sector size on snap-01 #1140

Open Firefishy opened 3 months ago

Firefishy commented 3 months ago

The NVMe disks in snap-01 are using 512 sectors instead of 4096 byte sectors. The sector size should likely be changed if boosts performance.

LBA Format  0 : Metadata Size: 0   bytes - Data Size: 512 bytes - Relative Performance: 0 Best (in use)
LBA Format  1 : Metadata Size: 0   bytes - Data Size: 4096 bytes - Relative Performance: 0 Best

Changing the disk sector size formats the disk, the sector size needs to be carefully coordinated with mdadm.

Outline steps:

  1. Confirm array idle via /proc/mdstats
  2. mdadm --fail
  3. mdadm --remove
  4. nvme format --lbaf=1
  5. mdadm --add

(repeat disk by disk across all 9 disks)

Firefishy commented 3 months ago

Related #1139