Simplify the Args situation

[axch]

The Venture source currently contains seven (!) Python classes whose name ends in the token Args:

• Args, in node.py, both defines (loosely) the interface of the object that Lite SPs expect passed in, and implements that interface as a "trace view".
• ForeignArgs, in foreign.py, is meant to be constructible by ForeignLitePSP from easily serialized information, as produced by Puma when calling Lite SPs.
• FixedValueArgs, in node.py, is a wrapper around an arbitrary Args that overrides its operandValues (and, optionally, operandNodes and spaux) fields to report given values. This is used to implement the assess SP and the DefaultVariationalLKernel.
• UnwrappingArgs, in sp.py, is a wrapper around an arbitrary Args that advises its operandValues method, applying a given venture->Python value transformation. This is used to implement TypedPSP.
• RequestArgs, in backend/untraced/evaluator.py, is used by the untraced interpreter to call Lite RequestPSPs. It is also constructed from whole cloth, but with venture.untraced.Node objects rather than venture values directly.
• OutputArgs, in backend/untraced/evaluator.py, is used by the untraced interpreter to call Lite OutputPSPs. It is a subclass of RequestArgs that also includes results of evaluating any requests made by the SP.
• BogusArgs, in test/randomized.py, is used by the test suite to directly call methods of Lite OutputPSPs without creating any kind of trace or nodes at all. It is constructed directly from the values and aux that are to be passed in.

One of the causes for this proliferation of Argh is that officially, the SP interface is supposed to permit SPs to obtain all sorts of information about their evaluation context from the interpreter:

• The actual arguments with which the SP is called
• The identities of those arguments (their nodes) (used primarily by parametrically polymorphic PSPs, like the RequestPSP in the implementation of lambda and the maker corresponding to mem)
• The SP's saved auxiliary state (spaux)
• If an AAA maker, the saved auxiliary state of the made SP (madeSPAux)
• The content of any evaluation requests the SP's requester stage may have made (used only by the fix SP?)
• The results of any evaluation requests the SP's requester stage may have made (used trivially by ESRRefOutputPSP and nontrivially by mapv and regress_mem)
• The identities (nodes) of those results (used only by fix and regress_mem?)
• The lexical environment of the application (used to implement lambda, get_current_environment, and fix)
• The identity (node) of the application itself (used mainly as a unique key to prevent auto-caching of requests that are supposed to be generative)
• The pseudo-random number generator to use for making any stochastic choices (mutable reference).

Most SPs, however, only care about the actual arguments and the prng, with a reasonable minority that also cares about the spaux and madeSPAux. The other stuff is both rarely needed and not always (readily) available, hence the alternatives.

The thing to do seems to be to

• [x] Define the Args interface as an interface, as part of the SP definition (as opposed to the Lite backend proper). Perhaps in sp.py?
• [x] Supply a lightweight generic implementation thereof (or a couple of them?), also as part of the SP interface, supplanting BogusArgs, ForeignArgs.
  • The maximally lightweight thing would not expose the Args instance to the client at all, but convert a desired SP method to a Python function directly. Something like sp_simulate(sp) -> lambda *args, seed=None, spaux=None, madeSPAux=None: just do it.
• [x] Supply a sufficient suite of generic combinators on Args objects, supplanting UnwrappingArgs and FixedValueArgs.
• [ ] Is there a unifying theory of RequestArgs and OutputArgs, or should we leave those as is?
• [x] Rename Args to TraceArgs or something, emphasizing that this class is relatively heavy-weight, and specific to running SPs at locations in PETs.

A somewhat higher-achiever intervention would consist of separating "normal" SPs from "magic" ones (what John Shutt called "operators" when specifying Kernel), and not even passing all the bizarre contextual information to "normal" SPs. Perhaps also disallowing "magic" SPs from being added to the system without patching the source. Besides simplifying the interface to "normal" SPs, this would have the added benefit that a future compiler can treat the set of "magic" SPs as closed and reason about each in depth, without having to be concerned about maintaining the ability to meet the "magic" interface for arbitrary user-addable SPs.

Thoughts?

[riastradh-probcomp]

How about 'Invocation' or something instead of 'Args'? 'Args' suggest it contains arguments, but it does much more than that.

Separating magic and normal SPs sounds good to me. I don't like magic!

[axch]

If we're going to rename the thing, I was thinking "Context" or "CallContext" or something. I balked at proposing that right away, because all the local variable names for it would need to be changed too.

[lenaqr]

In my Mite branch I have already elected to use the name "Context" for a different object... https://github.com/probcomp/Venturecxx/blob/mite-rev3/backend/mite/evaluator.py#L27

not that this isn't amenable to change.

[axch]

Progress update:

• Defined and (somewhat) documented the interface of Args as IArgs in psp.py.
• Added MockArgs in sp_use.py, replacing BogusArgs and ForeignArgs.
• Abstracted common use cases of MockArgs as the simulate, logDensity, and gradientOfLogDensity functions in sp_use.py.
• Added ReplacingArgs and RemappingArgs in sp_use.py, replacing FixedValueArgs and UnwrappingArgs, respectively.
• Renamed Args to TraceNodeArgs.

Justification for the remaining Args classes:

• MockArgs is an IArgs that presumes no tracing context (and does not supply argument identities).
• ReplacingArgs and RemappingArgs are IArgs transformers that preserve the context information.
• RequestArgs and OutputArgs are IArgs that supply the tracing context of the untraced evaluator. As such, they supply identities for the arguments as untraced.Node objects. There are two of them as a stylistic choice, one for RequestPSPs and one for OutputPSPs.
• TraceNodeArgs is the IArgs that started it all, supplying PET tracing context. As such, it supplies identities for the arguments as trace.Node objects.

Outstanding desiderata:

• Abstract the commonalities between simulate, logDensity, and gradientOfLogDensity. This is impeded by needing to figure out how to take an unknown number of arguments and a keyword argument in Python.
• RequestArgs and OutputArgs do not supply spaux or madeSPAux fields. If we were to augment them to do this, which we want anyway (#146), then they would be strictly more functional than MockArgs (and could perhaps replace it).