Support glob inputs and variable numbers of outputs

[theotherphil]

Fixes https://github.com/theotherphil/imagecli/issues/5

Example command lines cargo run --release -- -v -i '*.png' -o 'flipped_$N.png' -p 'MAP hflip' cargo run --release -- -v -i robin.png -o 'tile_$N.png' -p 'tile 100 100'

TODO:

• [x] Re-use the input and output handling from lib::process in InstantiatedExample::run.
• [x] Document globs in inputs and variables in outputs.
• [x] Remove calls to unwrap when loading inputs.
• [x] Think about having an option to run a pipeline independently on each input image. It's too easy to forget to use MAP if you're using a glob input. (Edit: not going to do as part of this PR)
• [x] Replace $ with something that doesn't require quoting to avoid being eaten by the shell.
• [x] Consider adding support for defining the output path in terms of the input path. e.g. foo.png -> foo_flipped.png. This requires output images to correspond to the input image with the same index to produce sensible looking results, but this is fine - we can just document this behaviour. (Edit: not going to do as part of this PR)
• [x] Update CHANGELOG.

cc @clbarnes, who might have some use cases this doesn't cover.

[clbarnes]

Nothing in particular I'd add - it would be nice to be able to specify a width for the numbers with zero-padding, so you could use a rust-like "flipped_{n:03}.png". That also avoids the $, although it possibly still needs quoting.

In terms of future additions:

Think about having an option to run a pipeline independently on each input image. It's too easy to forget to use MAP if you're using a glob input.

IMO this would be somewhat helpful in terms of ergonomics, but if we're going down that route, it would be an excellent opportunity for parallelisation.

Consider adding support for defining the output path in terms of the input path

That would be helpful, especially in the parallel case above. This would require some thought, though, as every image on the stack would need to carry some metadata with it - primarily the DAG of operations above it, of which the initial read would be one. That would give the choice of a lot more format options in the output name, too - source name, x and y index of tiles, operation list and so on.

[theotherphil]

Nothing in particular I'd add - it would be nice to be able to specify a width for the numbers with zero-padding, so you could use a rust-like "flipped_{n:03}.png". That also avoids the $, although it possibly still needs quoting.

Good idea - I'll do this.

In terms of future additions:

Think about having an option to run a pipeline independently on each input image. It's too easy to forget to use MAP if you're using a glob input.

IMO this would be somewhat helpful in terms of ergonomics, but if we're going down that route, it would be an excellent opportunity for parallelisation.

We could add a parallel fast path for MAP too if we can prove that there's no data dependency between the operators, but I'm not going to try this just yet.

Consider adding support for defining the output path in terms of the input path

That would be helpful, especially in the parallel case above. This would require some thought, though, as every image on the stack would need to carry some metadata with it - primarily the DAG of operations above it, of which the initial read would be one. That would give the choice of a lot more format options in the output name, too - source name, x and y index of tiles, operation list and so on.

I was thinking of just zipping input names and output files, as otherwise this gets a bit too complicated. I'll just go with supporting {n:k} for now.

[clbarnes]

We could add a parallel fast path for MAP too if we can prove that there's no data dependency between the operators, but I'm not going to try this just yet.

Yes, adding any parallelisation at all will probably be a big job. But once it's in in any form, it would be cool to envision a stack of Futures instead of realised images, which basically resolves as required, and in parallel where possible: let tokio handle the dependency graph. Then individual operations (like tiling) could also be done in parallel where appropriate. However, given that a lot of image works on mutable views and references (even when we're not mutating them), the ownership stuff will probably tricky.

I was thinking of just zipping input names and output files, as otherwise this gets a bit too complicated. I'll just go with supporting {n:k} for now.

Agreed, that would be a good first pass without the operation DAG requirement!

All this is out of scope for this PR, though. :smile: