libsubprocess: support stdout flow control

flux-framework / flux-core

core services for the Flux resource management framework

GNU Lesser General Public License v3.0

168 stars 50 forks source link

Open chu11 opened 1 month ago

chu11 commented 1 month ago

Separating this from #6291 as its a different beast and different design, and at the moment not an issue hit by users

chu11 commented 1 month ago

note some conversation from https://github.com/flux-framework/flux-core/pull/6281 and the overflow output callback in remote_output_buffered()

garlick commented 1 month ago

See flux-framework/rfc#427 for a proposed protocol.

chu11 commented 2 weeks ago

Was pondering this and the need.

with the unbuf support, it isn't needed for a large number of use cases now (but would be needed for flux-exec which doesn't use unbuf)
without this support, we probably gotta keep the buffer sizes large at 4 meg #6385

I think we care about #6385 a lot b/c 4 megs is a good chunk of memory or people are launching alot of processes