search

tomerfiliba / plumbum

Plumbum: Shell Combinators
https://plumbum.readthedocs.io
MIT License
2.79k stars 182 forks source link

Fix stalling in Pipeline command #632

Closed astaric closed 9 months ago

astaric commented 1 year ago

When a command (other than the first) in a pipeline wrote more than 64k to the stderr, and the output was consumed with the iter_lines function, the whole pipeline stalled. Fixed by reading the output of all commands where either stdout or stderr was set to PIPE.

This is the same issue as described in https://github.com/tomerfiliba/plumbum/issues/548

Same problem still exists if command is executed with run(), but that code calls proc.communicate to read stderr/stdout and I do not know how to fix that.

coveralls commented 9 months ago

Coverage Status

coverage: 84.548%. first build when pulling 3acc051c55b71ee646648df502212691dee36a29 on astaric:pipeline-stalling into d3ea8e3f1214538727da4cda88ae4c7b37b09623 on tomerfiliba:master.

vient commented 9 months ago

@henryiii do you plan to release new version soon? Stalling pipelines is something we've bumped in several times, this fix seems important.

I don't know how new releases are made, but if there is anything I can help with, I am ready :)

henryiii commented 9 months ago

Yeah, I'm planning one pretty soon. I want to get a release out then drop Python 3.6.

desertfury commented 4 months ago

Hi @henryiii ! I wonder have you shipped release that contains this particular fix yet? Thanks!