jojothewhal3 / fabricate

Automatically exported from code.google.com/p/fabricate
0 stars 0 forks source link

Compress redundant data in .deps file (or just gzip it) to reduce size #18

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
As per Michael Haggerty
http://groups.google.com/group/fabricate-users/browse_thread/thread/64670b732d07
1b75
-- the .deps file is both huge and contains lots of redundant info for
larger builds.

Consider compressing the redundant data somehow -- a simple gzip might be
the simplest way, and for a 17MB file this would actually speed things up,
because gzipping is faster than writing big files.

There are other, perhaps more code-complex ways to compress this data too.
But if gzip is chosen, it'll mean the .deps is no longer human-readable, so
have an no-zip override for debugging (but how often does one read their
.deps file?).

Original issue reported on code.google.com by benh...@gmail.com on 7 Aug 2009 at 2:12

GoogleCodeExporter commented 9 years ago

Original comment by benh...@gmail.com on 7 Aug 2009 at 2:13

GoogleCodeExporter commented 9 years ago
See especially Michael Haggerty's comments about "normalizing the data" in this
message: 
http://groups.google.com/group/fabricate-users/msg/b3d5e3466559b416?hl=en

Compressing redundant data in the .deps file and reducing this number of MD5s 
we need
to do as per Issue 17 are definitely related. Assuming it's not 
ultra-complicated,
this would be a better solution than gzipping.

Original comment by benh...@gmail.com on 10 Aug 2009 at 9:53