search

colcon / colcon-core

Command line tool to build sets of software packages
http://colcon.readthedocs.io
Apache License 2.0
99 stars 44 forks source link

Colcon hangs when stdout / stderr is redirected and output line is longer than default buffer size #624

Open vcomito-apexai opened 4 months ago

vcomito-apexai commented 4 months ago

I have to add a reproducer use case, but colcon seems to hang when a subprocess (e.g. cmake) writes a single stdout/stderr line longer than the default buffer size.

This is probably due to subprocess and PIPE which will hang when no newline is found in the buffer.

Increasing the buffersize in https://github.com/colcon/colcon-core/blob/master/colcon_core/subprocess.py#L159 seems to help

I will add a reproducer

cottsay commented 4 months ago

Just how long are we talking?

On my Fedora 39 machine, I wasn't able to get a hang with a 4 million character long line. It doesn't even seem to slow the build down, though gnome-terminal goes out to lunch if I try to scroll through the output.

vcomito-apexai commented 4 months ago

Just how long are we talking?

On my Fedora 39 machine, I wasn't able to get a hang with a 4 million character long line. It doesn't even seem to slow the build down, though gnome-terminal goes out to lunch if I try to scroll through the output.

iirc 100'000 characters. Also, apparently, the default size is 64 KiB. Sorry for not adding further instructions for reproducing this yet.

It doesn't even seem to slow the build down, though gnome-terminal goes out to lunch if I try to scroll through the output.

Was the output redirected to a file? I don't know exactly how this indirectly changes the behavior of the PIPE of the children processes

colcon ... > stdout.txt