rainwoodman
commented
8 years ago I think you shall make sure at least 1e4 ~1e5 items per file.
Otherwise, we may need to record both numbers; it's a measure of storage inefficiency of bigfile.
Open mscheucher opened 8 years ago
rainwoodman
commented
8 years ago I think you shall make sure at least 1e4 ~1e5 items per file.
Otherwise, we may need to record both numbers; it's a measure of storage inefficiency of bigfile.
@rainwoodman : Regarding my tests with iosim on COMA: We talked about a maximum size for all files in a run of around 500GB on COMA. are we talking of the sum of the actual sizes of the files (ls -lh) OR the amount of space on disk it all takes (du -ah)? I know it shouldn't be too different for huge files, but when I play around with a lot of files and items * width / files of around 1000, it becomes important. Or shall I just make sure that there are at least 10^4 - 10^5 items per file??