Noticed while processing a large file that it was going slow. Dug into the code and found that we were reading the whole file upfront exceeding the memory limit of my machine. Instead we can yield the object and let the processing happen line by line.
M0r13n
commented
1 year ago
Noticed while processing a large file that it was going slow. Dug into the code and found that we were reading the whole file upfront exceeding the memory limit of my machine. Instead we can yield the object and let the processing happen line by line.