I realize this project is nothing short of dead right now, halted to say the least. Still, I cling to the hope that someone will take the mantle and start working on its core code this new year. The idea is just as good as the first day, so having said that, let me propose another issue that needs to be addressed before releasing the first version.
All pre-processed and reordered data needs to be actually compressed before writing it to disk. We have considered some novel algorithms, but given the fact that none of them seem to be production-ready yet, I say lzma2 is as good as any, and battle-proven. Even without all the fancy new methods and algorithms for pre-processing, a Fairytale archive compressed with it should be about as small as a .7z, and smaller than a .zip, .zipx, maybe even .rar
OTOH, I noticed most of the people involved in data compression likes to work with paq* algorithms, context mixing and such. It's been said that paq8px internal structure doesn't provide all that's needed to produce the most efficient archive, so maybe this could be the next playground for neural networks, after porting the code of paq8px...
I realize this project is nothing short of dead right now, halted to say the least. Still, I cling to the hope that someone will take the mantle and start working on its core code this new year. The idea is just as good as the first day, so having said that, let me propose another issue that needs to be addressed before releasing the first version.
All pre-processed and reordered data needs to be actually compressed before writing it to disk. We have considered some novel algorithms, but given the fact that none of them seem to be production-ready yet, I say lzma2 is as good as any, and battle-proven. Even without all the fancy new methods and algorithms for pre-processing, a Fairytale archive compressed with it should be about as small as a .7z, and smaller than a .zip, .zipx, maybe even .rar
OTOH, I noticed most of the people involved in data compression likes to work with paq* algorithms, context mixing and such. It's been said that paq8px internal structure doesn't provide all that's needed to produce the most efficient archive, so maybe this could be the next playground for neural networks, after porting the code of paq8px...
Any thoughts about this?