To do list and help wanted

[traversc]

This issue is intended as a place to keep track of things I want to change or look at in future versions:

To do list:

• Refactor common functions (e.g. header read/write functions)
• Better error handling (file read check, compression check, RAII for PROTECT
• Can we use pipes as another option instead of just files? I.e., like the pipe function in R. Possibly with the boost library
• Other compression algorithms? In ver 0.16.1, Google's Brotli did not perform better than zstd
• Evaluate zstd's zstd_compressBlock function (in ver. 1.4.0)
• Support for R alt-rep aware serialization (Does this need to be R version aware? R 3.5 vs 3.6)
• Support for fast serialization of S4 objects
• Benchmarking/parameter optimization
• Automatic detection of SIMD in configure file
• Configure file for Windows possible?
• Evaluate other byte/bit shuffling routines, e.g.: -- https://github.com/kiyo-masui/bitshuffle -- https://github.com/powturbo/TurboPFor

On the off chance someone is reading this:

If you think this project is interesting and would possibly like to collaborate, please get in touch! I am especially looking for someone with expertise in C++11 concurrency and/or expertise in R alt-rep and R internals.