Add a `--wait` flag as global command

[whyrusleeping]

It was discussed in irc that it would be very nice to be able to start a command but not wait for it to return. If we implement ipfs diag cmds cancel <job> and ipfs diag cmds wait <job> we would have all the tools we need to deal with 'backgrounded' commands.

cc @pjz

[Kubuxu]

We would also have to return job number somehow.

[skorokithakis]

This (or rather, not this, but what it implies) is crucial for us in the development of Eternum (an IPFS pinning service). Our problem is that, currently, pinning things launches a task that may just time out if the file is too long, so there's no way to reliably charge for pinning. if we charge after the file has been downloaded, we might spend a month downloading a huge file without charging anything. If we charge before, we might be charging for a transfer that will time out or be interrupted at some point.

Also, if the user unpins the file before it's finished pinning, the pinning doesn't get cancelled. A job system for pinning files would solve all our problems, as then they would become job management problems instead of having to manage everything ourselves.

Basically, if pinning worked like a torrent client does, with starting/pausing/stopping/deleting, it would be ideal for us. As it stands, we have a very large problem even delivering on our promised services.

[forstmeier]

Hi, all! I saw that this related to the feature requested in #4041 but I wasn't sure of it's status - are these commands currently being worked on or does it still need to be built?

[whyrusleeping]

@forstmeier this still needs to be built. There is a small amount of design still yet to be done around the UX and API of this, but it should be pretty straightforward, all the hard bits and pieces are pretty much there already.

The main questions are "what do we name the flag?" and "how do we return the job ID?"

[forstmeier]

@whyrusleeping Cool! So I understand (and correct me where I'm wrong):

• The overall goal here is to allow commands to be non-blocking and a global flag will be added to allow that to happen. In order to support that, additional commands are needed to control those processes moved to the background (listed below).
  1. returning running command/job IDs (like the Linux jobs command)
  2. cancelling running background jobs by their job ID (#3115)
  3. a wait command (mentioned in #3116 but I need some clarification on it)

Let me know if I'm missing something in that list.

The main questions are "what do we name the flag?" and "how do we return the job ID?"

To address your questions while thinking out loud:

• Linux has the & option which allow the terminal to run the command in the background but for more verbosity maybe --async? I didn't see anything like that already when running ipfs commands but maybe it's not preferred.
• The ipfs diag cmds -v returns all in/active commands that have been run on that daemon - maybe adding another flag (-a, --active) just returns the ones running and the ID returned with -v can be used for cancelling/etc.? Just thoughts and I'd be happy to start poking at this if that sounds good. :wrench:

[whyrusleeping]

a wait command (mentioned in #3116 but I need some clarification on it)

This would basically be 'block until this job is complete'

Linux has the & option which allow the terminal to run the command in the background but for more verbosity maybe --async?

I like --async. That would work for me.

maybe adding another flag (-a, --active) just returns the ones running and the ID returned with -v can be used for cancelling/etc.

That can (and will) be one way to get the id of a command/job in order to cancel it, but I think its important to be able to get the job ID from each command you start, in case you run many commands that are similar, it may become hard to disambiguate them through the 'ps -a' type interface of ipfs diag cmds.

In any case, feel free to start poking at this :) My only ask is that you propose your approaches here (or in a separate issue) for some light discussion before starting to hack on things. More often than not some upfront discussion of approach would have saved a lot of contributor and reviewer time around here.

[forstmeier]

I'll definitely drop my plans/ideas into this and the other relevant issue comments so that everyone is in the loop. :smiley:

but I think its important to be able to get the job ID from each command you start

For this I'd assume that either 1) commands would need to generate the ID as a default or 2) --async would generate it (and output it to the command line) when included as a flag on the given command. Another option could be to have the -a, --active flag on the ipfs diag cmds produce additional metadata about the currently active background commands (e.g. showing the name of the file being pinned). Thoughts?

Thanks for the clarification/guidance!