This is a major rewrite of the LibraryContainer class, more specifically the functionality for parallel I/O. Additionally, I've taken the opportunity to clean up the entire library.cpp and library.h file. This includes the refactoring of the other classes in those files (RecordBlock and ReadLibrary).
The rewrite of the LibraryContainer was necessary because only limited parallelism across RecordBlocks was allowed in the previous version. More specifically, a worker thread could not commit a chunk to a RecordBlock if the previous RecordBlock was not yet completed. In practice, handling a new RecordBlock caused thread synchronization until the last chunks from the old block were completed.
In the current version, parallelism across RecordBlocks is unlimited. The only restriction is the number of RecordBlocks simulateneously held in memory. This is controlled by a #define in global.h. It is currently set to two, which should be fine unless some chunks take excessively long.
Please revise and test thoroughly. I've tested the code with valgrind (no leaks, no helgrind issues) and for functionality (output is identical to master for S. Aureus).
This is a major rewrite of the LibraryContainer class, more specifically the functionality for parallel I/O. Additionally, I've taken the opportunity to clean up the entire library.cpp and library.h file. This includes the refactoring of the other classes in those files (RecordBlock and ReadLibrary).
The rewrite of the LibraryContainer was necessary because only limited parallelism across RecordBlocks was allowed in the previous version. More specifically, a worker thread could not commit a chunk to a RecordBlock if the previous RecordBlock was not yet completed. In practice, handling a new RecordBlock caused thread synchronization until the last chunks from the old block were completed.
In the current version, parallelism across RecordBlocks is unlimited. The only restriction is the number of RecordBlocks simulateneously held in memory. This is controlled by a #define in global.h. It is currently set to two, which should be fine unless some chunks take excessively long.
Please revise and test thoroughly. I've tested the code with valgrind (no leaks, no helgrind issues) and for functionality (output is identical to master for S. Aureus).