model output directory management

[jzanzig]

1. delete failed model runs
2. ignore metrics in directory name

[potash]

65f682627eb408f3f70c0abbfcee836a22d8ae27 fixes the second issue

[potash]

Could have drain --autoclean delete any step output that is:

• missing a target file (i.e. never successfully ran)
• stale target, which can be determined by trying to construct the given object:
  • if it fails to construct then something is wrong with the args (signature changed, etc.)
  • if it constructs, then collect all source code timestamps for all inputs and ensure target timestamp is newer.

need to be careful though. could have an interactive mode or dump a list of steps eligible for cleanup.

[potash]

Could have drain --keep delete anything that's not not in the specified workflow(s).