Makeflow file name interpolation missing for wrapper

[batrick]

Sometime after ada96bf2c0ad4fc575564680d15de9ee5b0befcb, Makeflow lost support for file name interpolation in wrapper output/input. For example:

makeflow --wrapper='{} > stdout.$NODE 2> stderr.$NODE' --wrapper-output='stdout.$NODE' --wrapper-output='stderr.$NODE' makeflow/example/example.makeflow -T chirp --working-dir=chirp://localhost:10000/

This works in ada96bf2c0ad4fc575564680d15de9ee5b0befcb (which my confuga branch is currently based on), but it does not work in master.

Also, support for $NODE and $RULE is also missing. Without these variables, it's hard to have per-rule files with the wrapper command, right?

By the way, the Makeflow man page and documentation requires expansion on: curly braces {} are replaced by the actual command. My interpretation of the docs is that the wrapper command is only prepended to the actual command.

[dthain]

@batrick - I think you picked a commit right in middle of the development of the wrapper feature, and so what you have isn't quite complete and in fact didn't pass all tests. The head of master is complete, but behaves in a different way than you might expect.

After working on the code for a while, it became clear to me that $-substitutions in the wrapper are not a workable idea. The problem is that $-substitutions are macros that occur at parse time, while the wrapper (by definition) is applied to the rule just before execution. At that point, all the macros have already been applied, and performing another set of substitutions would introduce a whole additional level of escapes and complication.

The solution is that %% for the node number is the only substitution allowed in the wrapper command or files, and this noted in the wrapper document.

See 29dde458 and 444bae7

@btovar @charleszheng44

[batrick]

That makes sense. I think all that's left is getting the docs in sync with the code.