MarFS provides a scalable near-POSIX file system by using one or more POSIX file systems as a scalable metadata component and one or more data stores (object, file, etc) as a scalable data component.
We could support special filenames (e.g. in a namespace that has a new kind of "testing" indicator), or maybe special xattrs that can't be installed by regular users. When handed to marfs, these filenames or xattrs would cause flags to get set in a _testmode field in the corresponding FileHandle.
Among other things, our code could then use these flags to trigger various simulated timing issues, when interacting with object-servers. This would let us systematically test many situations where e.g. a server stops sending to curl during a GET (or receiving during a PUT). We could systematically provoke every kind of timeout condition that our code recognizes.
We can test normal corner cases with regular test code. The point of these would be to add support for testing specific situations in the internals, which are otherwise difficult to recreate without going into the debugger and creating them explicitly. Some of these could even be probabilistic, or time-delayed, so that we can get a heavy load of traffic going before they occur. Conceivably, we could even set things up to allow multiple independent threads to coordinate in "bad" ways.
We could support special filenames (e.g. in a namespace that has a new kind of "testing" indicator), or maybe special xattrs that can't be installed by regular users. When handed to marfs, these filenames or xattrs would cause flags to get set in a _testmode field in the corresponding FileHandle.
Among other things, our code could then use these flags to trigger various simulated timing issues, when interacting with object-servers. This would let us systematically test many situations where e.g. a server stops sending to curl during a GET (or receiving during a PUT). We could systematically provoke every kind of timeout condition that our code recognizes.
We can test normal corner cases with regular test code. The point of these would be to add support for testing specific situations in the internals, which are otherwise difficult to recreate without going into the debugger and creating them explicitly. Some of these could even be probabilistic, or time-delayed, so that we can get a heavy load of traffic going before they occur. Conceivably, we could even set things up to allow multiple independent threads to coordinate in "bad" ways.