v2 Part 2 1/2 (or 3): Scoped, inheritable and dynamic configuration handling changes

[svanoort]

Why:

• For users it's helpful to define a variable and use it across a whole testset, or set it ONCE at the command line
• Currently to add a new option at command line, testset config and test level, you need to add code in 8 places (parsing in 3, variables in 3 objects, actual use plus docs... and with tests for each). Fixing the parsing in part 1 will fix ONE part of this (reducing it to parsing in one place, docs, and variables in 3 places). This reduces it to 3 places total (docs, parsing, and) which is about as good as it gets.
• By allowing automatic templating, everything is dynamic if needed (much more powerful), easy to add pluggable templating
• Greatly reduces codebase complexity, I hope and increases flexibility (similar to contexts and the validator registries).

  Problems

• Current gen of extensions cannot consume this. New extension APIs will need to be defined to use it (probably want to add kwargs/args support so extensions can grow organically)
• Need to clearly document scoping rules and parsing rules with regards to scoping

  What:

• Templating: any configuration element can be defined as dynamic, allow more than one templating option
• Future enhancement: allow use of a pre-compile pass on a run to optimize away un-needed dynamic-ness for performance

  Nested context structure

I've got full architecture notes on this now, but the gist is:

• Configuration context will be passed in as a list of dictionaries, with each dictionary being a scope
• Current scopes will be: command-line, testset, test, with priority increasing towards the right
• Overall, configuration will be a tree (command line, testsets, and tests under testsets)
• There will be a number of reserved keywords for configuration elements, to support built-in and special items

  Contexts

• Configuration context (generated during test parsing) - may be specially modified in tests, but discouraged -- SCOPED
• Execution context (caching, information needed for run state, optimization info) - SCOPED
• Test context - variables in tests, etc - redefinable easily - NOT SCOPED, GLOBAL

(This will be edited some after the parsing rearchitecting in https://github.com/svanoort/pyresttest/issues/75 is done)

Link on templating performance: http://stackoverflow.com/questions/1324238/what-is-the-fastest-template-system-for-python

[svanoort]

CAVEAT: Some fields can accumulate values additively, for example multiple headers can be added.