Writeboost on top of VDO

[jvinolas]

We tried to build db-writeboost cache on top of vdo (https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/deduplicating_and_compressing_logical_volumes_on_rhel/introduction-to-vdo-on-lvm_deduplicating-and-compressing-logical-volumes-on-rhel) deduplication.

When creating the writeboost cache, after few minutes, the RAM on the server, that was being used at a constant 36%, peak to 100% instantly, triggering the OOM.

The creation was:

  163  BACKING=/dev/mapper/vg_vdo-vdo_lv0
  164  CACHE=/dev/md1
  165  sz=`blockdev --getsz ${BACKING}`
  166  dmsetup create wbvdo --table "0 $sz writeboost $BACKING $CACHE 2 writeback_threshold 70"

The devices are setup as following:

Backing Device: VDO over a raid

sdm                           8:192  0  6.3T  0 disk  
└─sdm1                        8:193  0  6.3T  0 part  
  └─md2                       9:2    0 50.8T  0 raid0 
    └─vg_vdo-vpool0_vdata   253:0    0   40T  0 lvm   
      └─vg_vdo-vpool0-vpool 253:1    0  500T  0 lvm   
        └─vg_vdo-vdo_lv0    253:2    0  500T  0 lvm

Device                   Size      Used Available Use% Space saving%
vg_vdo-vpool0-vpool     40.0T    522.7G     39.5T   1%            4%

Caching device: raid

sdc                           8:32   0    1T  0 disk  
└─sdc1                        8:33   0 1024G  0 part  
  └─md1                       9:1    0    4T  0 raid0

And the writeboost packages are the Ubuntu Jammy repository one's:

dm-writeboost-dkms/jammy-updates,now 2.2.13-1ubuntu3 all [installed]
  log-structured caching for Linux

writeboost/jammy,now 1.20170616-1.1 all [installed]
  utility to manage mappings of dm-writeboost devices

Is this scenario expected to work?

[akiradeveloper]

How much memory does your server have?

[jvinolas]

64 GB, only 22GB used till the hung that peaks instantly to consume the whole ram.

UPDATE: I've implemented open-cas with the same system and worked, so I assume it should work with writeboost also.

[akiradeveloper]

According to wiki, writeboost consumes roughly 1% of the cache device size for the in-memory metadata. In your case, the cache device is 4TB so the system consumes 40GB RAM. I think this is the cause of your problem.