Data files are currently stored in non-nested MD5 directories. Contemporary operating systems cannot deal with hundreds of thousands of directories within a single directory. In order to alleviate the disk we can use a similar approach to Git: nest MD5 directories on a 2-character bases. For example: /1234567890abcdef/clean.nq.tar will become /12/34/56/78/90/ab/cd/ef/clean.nq.tar.
Data files are currently stored in non-nested MD5 directories. Contemporary operating systems cannot deal with hundreds of thousands of directories within a single directory. In order to alleviate the disk we can use a similar approach to Git: nest MD5 directories on a 2-character bases. For example:
/1234567890abcdef/clean.nq.tarwill become/12/34/56/78/90/ab/cd/ef/clean.nq.tar.