ajratner
commented
9 years ago @raphaelhoffmann Okay- doing the above (collecting while parsing / at intermediate points) might be good in general, but an easier solution: on ec-2 instance storage is not mounted by default! So e.g. with an m3.2xlarge instance we are writing to an ~8GB root drive, when we could (for the same price) configure it to have 160GB of SSD storage!
Fixing this now
It seems like both to avoid this issue & for general reasons we might want to collect parsed segments as we parse- will see if relatively easy way to do this