Open kdk1616 opened 4 years ago
Maybe a hotness-o-meter. That way infrequently accessed files can be serialized and zipped etc. . JSON is not an archival format
Maybe a hotness-o-meter. That way infrequently accessed files can be serialized and zipped etc. . JSON is not an archival format
JSON is good enough for archival. My largest export file is only 214 kilobytes, but this is 14.8K gzipped and 12K after Zstandard, which is also faster than gzip. If we choose to compress with one of these algorithms, the time it takes to compress would be so small that there would be no point triaging files and we might as well compress every one of the JSON files stored on the server.