m(easuring) buffer

AUTHOR: Thomas Maier-Komor e-mail: thomas@maier-komor.de homepage: http://www.maier-komor.de/mbuffer.html

LICENSE:

GNU GPLv3 (see file LICENSE for details)

DESCRIPTION:

mbuffer is a raplacement for buffer with additional functionality:

display of i/o speed

optional use of memory mapped i/o for huge buffer files

multithreaded instead of sharedmemory ipc

multi-volume support

auto-loader support

network support

buffer compatible command-line options

NETWORKING:

The included networking code is based on TCP/IP. To use it you should know the basics of TCP/IP. Use it only on a trusted LAN, as there is no builtin security.

TESTING:

to test the program I do the following:

tar cf - /usr | mbuffer | tar tf - > out

tar cf - /usr | mbuffer -t | tar tf - > out

ARCHITECTURES and PLATFORMS:

This software has been tested under the following operatingsystems:

• solaris 8 (SPARC and x86) and later
  • no known issues
  • do not put your temporary files on a tmpfs filesystem, as this is equivalent to a normal memory allocated buffer
• linux 2.2 (x86) and later
  • some pthread versions of linux seem to have problems with cancellation, causing a segmentation fault upon SIGINTR
• tru64-alpha
  • broken display
• FreeBSD 5.x (x86)

MacOS:

mbuffer does not compile and run on MacOS.

The reason for this is the lack of the sem_getvalue interface, which MacOS deliberately refuses to offer. The argumentation is that the returned value might be incorrect by the time it gets used, as the semaphore value is not locked.

This argumentation is irrelevant in the case of mbuffer, which uses the function to inspect the transient status, and thread synchronization is done on variables under mutal exclusion.

If anybody is willing to provide a patch that works around this limitation, I will consider its inclusion. As I don't have a MacOS machine, I have no intention of working on that topic.

64 Bit Buffers:

The buffer limit is sysconf(_SC_SEM_VALUE_MAX)*Blocksize. This usually limits the maximum number of blocks to 2G-1, because semaphores are implemented using an int, which is on most architectures a 32 Bit word. This limit is currently a non-issue, but might be resolved in a future release, by using condition variables.

If the total buffer size is 2GB or larger, you will need to compile mbuffer as a 64bit executable. This requires that you have a 64bit processor. Compiling with gcc you will need to pass -m64 as CFLAGS to configure. E.g.: env CFLAGS="-O -g -m64" ./configure For Sun's Studio Compiler the equivalent flag is -xarch=v9.

TODO:

fix: unknown bugs

FEEDBACK:

It is always nice to get feedback. If you encounter a problem or a bug, send me a note. Requests for enhancements are also welcome. (software@maier-komor.de)

DONATIONS:

If you like this software, and use it for production porposes in your company, please consider making a donation to support this work. You can donate via PayPal to the author's e-mail address: thomas@maier-komor.de