Feature request: stream the subprocess' stdout and stderr while it's running

[borestad]

Hi!

Really useful cli tool. I use this mostly for downloading large files in a cronjob. Something that I'm currently lacking is a strategy for streaming the progress to stdout. Both in interactive and in CI-scripts.

What are your thoughts about this?

[dimo414]

Thanks for filing, yes I'd like to implement this (for both stdout and stderr), similar to what fzf did. This is mentioned in the README and a few other places but there should be a tracking issue.

I think this is fairly straightforward to implement, but it involves spawning new threads to process stdout/stderr manaully and I've just avoided pulling it together :) I'll try to take a swing at it in the next few weeks, but if you or anyone else is interested in sending a PR it shouldn't be too much work.

[borestad]

Ah, I see. Unfortunately, Rust isn't my main language (although I can read the code). Otherwise you'd have a PR in notime.

As I see it, this together with a couple of more extra api's like "has" (to check if the value already exists) + "stats" with information about remaining ttl etc ...+ the ability to create a unique key instad of hashing the input ....would make me throw out almost all my current caching needs entirely! 🥰

[dimo414]

You might be interested in #24 and #19 as well in that case.

Would you mind starting a discussion thread with more details about how you might use bkt if the additional features you're envisioning existed? It would be very helpful to understand your desired use-cases better.

[borestad]

Hi, sorry for late response. Fever got in the way. Anyway, I wrote here to show my "memo" helper for streaming output.

https://github.com/dimo414/bkt/discussions/29#discussioncomment-5795865

I'll come back with other details of how I'm using bkt - but in the memo helper it's obvious that I need to use some "hacks" to acchieve:

• Check for an existing key
• Delete an existing key
• Show TTL left
• Create a unique key (to be able to delete it later)

[dimo414]

Thanks the the thoughts! I moved them to a separate thread so it's easier to discuss further. For invalidation specifically you can also weigh in on #5.

[dimo414]

For folks interested in this issue, PR #43 will implement streaming support. Please take a look at the PR if you have any feedback before it's merged. You can also grab binaries off the CI runs to test these changes out locally.

[dimo414]

@borestad if you have a minute to try out PR #43 I'd appreciate it! Planning to merge and cut a new release in the next few days.

[borestad]

@dimo414 I did a quick test with just a minor script like:

sleep 1
echo 1
sleep 1
echo 2
sleep 1
echo 3

....works perfect! This is really really great stuff 👍