tomerfiliba / plumbum

Plumbum: Shell Combinators
https://plumbum.readthedocs.io
MIT License
2.82k stars 184 forks source link

Iterate lines without caching whole program output #480

Open GoldsteinE opened 5 years ago

GoldsteinE commented 5 years ago

For now .popen().iter_lines() caches all of process output, so it's unusable for long-running processes. E. g.

from plumbum.cmd import yes
for _ in yes.popen().iter_lines():
    pass

will eventually eat of all memory. There should be an option to discard old output and consume memory only for current lines.

shinebayar-g commented 4 years ago

Is there any workaround?

andressommerhoff commented 1 month ago

Hi! any update on this?

henryiii commented 1 month ago

PR welcome if you have an idea.