Make evaluator run evaluation tasks/commands directly

[ghost]

(originally reported in Trac by @elehack on 2012-03-28 21:15:07)

Currently, the Groovy scripts for evaluators set up tasks which are then run by the eval runner.

We want to make the evaluation tasks run directly by the script, so that their output can be used & reacted upon. This will, in general, make the evaluator much more usable problematically, with negligible impact on simple scripts.

To do this:

• Turn tasks into Commands. Command extends Callable, throwing EvalCommandFailedException. Commands have setters & adders, like builders currently do, to set things up for their call() methods. So the crossfold builder, for example, just becomes a CrossfoldCommand, rather than having both CrossfoldTask and CrossfoldTaskBuilder. Builders still exist for building data types such as preference domains and CSV data sources. Command and Builder may be a slightly spurious distinction, but let's go with it for now.
• Make the Groovy machinery treat commands like builders (setting them up, calling their call() method, etc.). They can also support the [[javadoc(org.grouplens.lenskit.eval.config.ConfigDelegate)]] annotation.
• Provide a way for the eval script to get arguments. We'll want to see how Groovy generally does this, esp. with GroovyShell and GroovyScriptEngine. I then propose that the CLI runner take the script as its first non-option argument, and then pass all remaining arguments to the script. Then the script can do things like only run a few tasks if it wants.

Note: This issue has been automatically migrated from Bitbucket Created by grouplens on 2013-02-01T22:06:04.731500+00:00, last updated: 2013-02-01T22:35:27.848197+00:00

[ghost]

ekstrand on 2012-05-21 21:19:30 said:

This is now done & merged.

Note: This comment has been automatically migrated from Bitbucket Created by grouplens on 2013-02-01T22:06:05.440018+00:00, last updated: None

[ghost]

shilad sen https://www.google.com/accounts/o8/id?id=aitoawl-dxnionmrai01nscmtxfwwdxbne1j6dm on 2012-04-01 17:48:04 said:

I would love to have the third feature (access to command arguments).

Note: This comment has been automatically migrated from Bitbucket Created by grouplens on 2013-02-01T22:06:05.067535+00:00, last updated: None