search

chmln / sd

Intuitive find & replace CLI (sed alternative)
MIT License
5.61k stars 136 forks source link

Follow STDIN input #100

Open vtronko opened 3 years ago

vtronko commented 3 years ago

I expect sd to follow the STDIN, as it would do in sed. For example, I expect journalctl -u dnscrypt-proxy.service -f | sd machinename home to give consistent live output, as journalctl -u dnscrypt-proxy.service -f | sed '/s/machinename/home/ would. Instead, it just waits for EOF, due to handle.read_to_end(&mut buffer)?;

I'd suggest to use terminal buffering mode (flush buffer at newline), but that'd break multiline replacement.

This is an important usecase for log inspection routines with custom formatting.

jnnkB commented 3 years ago

This would be really cool and could perhaps be archived by adding an line by line mode.

sebastianblunt commented 2 years ago

Easy to replicate with (echo a && sleep 5 && echo a) | sd a b. Probably related to https://github.com/chmln/sd/issues/96

varenc commented 1 year ago

I'm also struggling with this issue. Usually the stdbuf or the unbuffer tools are able to force programs to be line buffered, but no luck using those for sd. (stdbuf relies on tweaking the version of libc that's loaded, but since sd is written in rust, I assume that's why it doesn't work).

When I want to view the live output of a series of pipes, I sadly have to switch back to using sed even though I vastly prefer sd. GNU Sed has an --unbuffered flag which instantly accomplishes this.

intelfx commented 1 year ago

This is not related to buffering. sd seems to work by reading whole stdin until EOF into memory prior to performing any work on it. Thus, sd it is not really a sed alternative as it is not "streaming".

brunetton commented 1 year ago

Same here (super useful comment, I know)

vtronko commented 1 year ago

This is not related to buffering. sd seems to work by reading whole stdin until EOF into memory prior to performing any work on it. Thus, sd it is not really a sed alternative as it is not "streaming".

Exactly why this is related to line-based buffering, the lack thereof…

vtronko commented 1 year ago

For what it's worth, I created a fix which works, but I need to create more tests for it. I will publish the code later.

borestad commented 10 months ago

Any progress on this? 🙂

corneliusroemer commented 7 months ago

It looks like @vtronko's commits on his fork work - I tested them locally. They need to be rebased onto the current main but otherwise looks good. @chmln can you have a look at: https://github.com/vtronko/sd/commit/fcd59ace3a41e97328079d95c052a0570733f75c?

CosmicHorrorDev commented 7 months ago

I know @dev-ardi has interest in making this change. The limiting factor has been maintainer time at the moment (aka I've been very very busy)