Right now when a file is processed in textkit, the entire file is loaded into memory for further processing.
This is probably the wrong thing to do in python.
For reading files and writing them, is there a general solution that can take advantage of python generators that doesn't destroy the understandability of the code - but makes the memory footprint of the functions smaller ?
Right now when a file is processed in
textkit, the entire file is loaded into memory for further processing.This is probably the wrong thing to do in python.
For reading files and writing them, is there a general solution that can take advantage of python generators that doesn't destroy the understandability of the code - but makes the memory footprint of the functions smaller ?