SSD wear when utilizing compression

[akseg73]

Is there some way to reduce the "chunk-size" for compressed blocks in order to reduce write amplification. "chunk-size" here would mean the smallest block of compressed data, which has to written individually. Depending upon the use case at hand, is it possible to reduce the chunk size to something closer to 16kb or less. Having a much larger chunk size can create problems in presence of frequent modifications and can severely degrade SSD life.

chunksize above may not mean the same thing as the option provided to lvcreate.

[akseg73]

Even if such a feature is not explicitly provided, i would imagine that the change should require modifying some constant somewhere and rebuilding the kernel modules and installing with those. If you can point me towards the header file that has the said constant i can make the change and rebuild and test with it

[lorelei-sakai]

The short answer is no. All compression is done in 4kb chunks and the granularity cannot be changed.

But to go into a little more depth, what write amplification are you talking about that you want to prevent? VDO generally will only write each unique block once (compressed or not). As phrased, the question doesn't seem to make sense for VDO.

The compression handling is mostly in packer.[hc]. That would be the place to start looking if you want to dig into this yourself.

[akseg73]

Thanks for the response. I was guessing the internal workings of VDO based upon how ZFS works. So if the file is divided up into contiguous compressed chunks, then modifying 4k within a chunk would lead to reevaluation of the compression of the whole chunk and lead to write amplification. This is a kind of usage in which modifications are expected to occur on compressed data. If you compress the data and it remains read only thereafter then it does not matter. On the other hand if you expect to modify compressed data, then you would have to recompress it. Now it becomes a matter of what other data is adjascent to the data you are compressing/uncompressing and what other data will end up sharing space with the data that you are recompressing.

So the question i am posing only makes sence in an environment where compressed data is frequently modified. If VDO does not work in such a case then offcourse write amplification makes no sense.

[lorelei-sakai]

I'm not familiar with the way ZFS works. VDO really only works on 4k blocks and doesn't use larger chunks for anything, so I think these amplification concerns don't affect VDO. If the compressed data is modified and rewritten, vdo does have to compress the new data, but it don't rewrite compressed data that hasn't been changed. Newly compressed fragments are combined with other newly compressed data and written as new data blocks.

[akseg73]

Thanks for the response. Then it appears to be quite different, it would also seem that this design maybe is more aligned with the natural bahavior of SSDs which do not perform update in place. On the other hand this design conceivably requires more book keeping to find out what is where and might end up adding to some write overhead. I would have to dig much further into the internals to determine if this really matters or you have some nifty ways to avoid that as well.

[lorelei-sakai]

Yes. VDO does have to track a fair amount of metadata and the write amplification concerns it has mostly center around making sure the metadata writes are manged efficiently. Generally the I/O saved by deduplication and compression will outweigh the metadata costs, but for workloads that don't have much duplicate data, VDO may not be a good choice.

[akseg73]

The data we have is highly compressible but we will turn off dedup, we don't have duplicate data. A use case where compression is the main goal, would that work with VDO?

[raeburn]

It's not the sort of case we test or tune for, but it should work fairly well. At least, assuming the data is compressible in small chunks, and not because, say, a 5kB string of binary data doesn't compress well itself but is repeated a lot with only minor variations -- then all of your compressibility would come from seeing those long strings of bits repeated a lot. Most compressible data I've seen compresses fine in smaller chunks (e.g., text, or small and highly structured binary records), so yes, I would expect that to work well with VDO.

Depending on your data, you might find it useful to tweak the compression code a little to specify a higher level of compression (and higher CPU utilization) if it improves the space savings. I believe the driver is hard-coded to use the default or fastest level of compression. Unfortunately(?) we don't have controls available for the compression level or alternate compression algorithms.

[akseg73]

What about a case where 4k bytes contain 30% contiguous empty space, that is an ideal case for compression. There is also a case where two pages (4k maybe) are identical with the exception of 50 bytes in this case i would imagine you would require dedup to be switched on, compression alone lz4 would not be able to figure this out.

Lets focus just on the case where every block of 4k written is 30% empty would that work reasonably well.... just to get a baseline understanding of what to expect