What was done: instead of creating pickle data before pickling, we create an iterator which was meant to yield small portions of data on demand.
While the approach seemed to be a good solution, it appears that both pickle and _pickle modules collect in memory whole dump (despite the way it's created) and then, in the end, flush everything to the output file.
An attempt to solve #102.
What was done: instead of creating pickle data before pickling, we create an iterator which was meant to yield small portions of data on demand.
While the approach seemed to be a good solution, it appears that both pickle and _pickle modules collect in memory whole dump (despite the way it's created) and then, in the end, flush everything to the output file.