shashi / FileTrees.jl

Parallel computing with a tree of files metaphor
http://shashi.biz/FileTrees.jl
Other
88 stars 6 forks source link

How to map multiple patterns in `mapsubtrees`? #74

Open singularitti opened 1 year ago

singularitti commented 1 year ago

Suppose I have a FileTree and two patterns: *.in for input files and *.out for output files. Usually, in the same folder, there will be two files: a *.in paired with a *.out. How can I match them at once in mapsubtrees? Like the following syntax:

mapsubtrees(tree, ("*.in", "*.out")) do (subtree1, subtree2)
    path(subtree1) => path(subtree2)
end
DrChainsaw commented 1 year ago

This seems like a very useful function to have. I wonder if it will be difficult to have consistent semantics for the non-happy path cases.

For example, what should it do if the number of matches is not balanced (i.e number of matches for *.in is not the same as number of matches for *.out)?

Here is an example of what I usually do when I need to combine multiple files in the same dir: https://github.com/shashi/FileTrees.jl/issues/51#issuecomment-802152231

It does not have as convenient syntax as your example, but one can add that if needed. It is basically a map but only for directories which don't have subdirectories. For you example, you can then pick out the files matching your patterns from the argument passed to f (knowing they are in the same directory) and apply whatever function you need.

Reason why I have not put in into FileTrees is exactly that does not have consistent behaviour for all cases this package is supposed to handle.

shashi commented 1 year ago

"*.(out|in)"? :)

shashi commented 1 year ago

Oh i see what you mean.

I think you could do something like get the tree for the output, rename ".out" to ".in", and then merge the original .in tree with a function to combine.