Implement filesystem fault-injections

[ligurio]

It would be nice to implement fault-injections applicable to operations with filesystem. They can be implemented using PetardFS or CharybdeFS. Both are FUSE filesystems with management layer to control injections (in PetardFS it is an XML config file and in CharybdeFS it is Thrift interface).

Possible errors:

• Incomplete writes to disk
• Corrupted writes to disk
• Disk drive runs out of space
• Delay on operations
• Failed operations with random errno

See also:

• Consider nemeses used in Foundation DB #27
• Add internal Tarantool ERR_INJs #26
• FUSE filesystems:
  • CharybdeFS
  • PetardFS
  • UnreliableFS
• device-mapper:
  • dm-flakey https://www.kernel.org/doc/Documentation/device-mapper/dm-flakey.txt
  • dm-delay https://www.kernel.org/doc/Documentation/device-mapper/delay.txt
  • dm-dust https://www.kernel.org/doc/Documentation/device-mapper/dm-dust.txt

Jepsen framework is lack of filesystem's nemeses support (see https://news.ycombinator.com/item?id=20166483):

Hi there! There's work to do on both, and I encourage you to contribute to core if you like! We could really use performance improvements on the checkers--Knossos and set-full are great, but much more expensive than they need to be. It'd be nice to have a robust mechanism for adjusting CLOCK_MONOTONIC & friends. I don't have a good way to inject filesystem level faults, like simulated node crashes where non-fsynced data is lost. jepsen.control is plagued by what I think are race conditions in Jsch, which I've never had the chance to dig in and fix.