LINBIT / drbd

LINBIT DRBD kernel module
https://docs.linbit.com/docs/users-guide-9.0/
GNU General Public License v2.0
587 stars 100 forks source link

Question: is it possible to increase DRBD sync speed ? #61

Closed ghevge closed 1 year ago

ghevge commented 1 year ago

Hi, I've just installed DRBD functionality in between 2 VMs (each with 22 CPU cores, 60 GB RAM and a 930 GB DRBD shared disk ). All good but I have a question regarding the sync speed. After I've starterd the initial disk synchronization, I can read network speeds up to 40 MB / s.

On each of my host servers I have 2 x 1 Gbps network cards bounded in balance-alb mode. On each server i have 2 x Xeon 2630V2 with 64GB RAM and 2 x 1 TB SSD in RAID 1 configuration. As virtualization env I use Proxmox 8. The Switch in between the nodes is a 48 x 1GB + 2 x 10GB upload links. There are no other VMs running on these servers other than the 2 VMs each on its one server.

Theoretically I should see network speeds up to 200MB/s but drbd sync speed seems to be stuck at 40 MB/s. On standard file transfers operations I can see regularly speeds in the range of 160 MB/s in between the VMs.

NetworkSpeedDuringInitialSync

 cat /proc/drbd
version: 8.4.11 (api:1/proto:86-101)
srcversion: 9B671CCC1F00886BA069043
 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
    ns:180441216 nr:0 dw:0 dr:180441216 al:16 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:794704668
        [==>.................] sync'ed: 18.6% (776076/952288)M
        finish: 5:27:02 speed: 40,480 (39,064) K/sec

Any idea how I can increase the DRBD sync speeds ?

Thanks

ghevge commented 1 year ago

I've managed to speed it up using the information in this post: https://serverfault.com/questions/740311/drbd-terrible-sync-performance-on-10gige