In my understanding, a user who is writing a Metaprob inf must specify two gen procedures: the "sampler" and the "scorer".
The sampler is a generative model which specifies the forward simulation process and the scorer is a proposal distribution which returns unbiased estimates of the log-marginal probability of constraints, as specified by an arbitrary combination of intervention and target traces.
The default scorer/proposal distribution is the prior, which is roughly how Metaprob elevates a "gen" into an "inf".
Based on discussion with @alex-lew (please weigh in if there are errors in this summary), it seems sensible to allow the user to write a scorer that knows how to compute only some combinations of target and intervention traces and ask Metaprob to revert to the prior to handle the rest. This case arises when a user wants to optimize for a particular conditional query that they know how to handle well, but does not care about obtaining good estimates on other queries.
The proposition is thus to write a "scorer" generator that takes in a partial scorer and fills in the rest. There needs to be some thought for how (i) how to design the interface (for example, the user can specify a list of conditionals that they know how to propose); and (ii) the algorithms needed to complete the user-specified sketch (by reverting to the prior) in a probabilistically coherent way.
Marking as "research" which could serve as a good starter project.
In my understanding, a user who is writing a Metaprob
infmust specify twogenprocedures: the "sampler" and the "scorer".The sampler is a generative model which specifies the forward simulation process and the scorer is a proposal distribution which returns unbiased estimates of the log-marginal probability of constraints, as specified by an arbitrary combination of intervention and target traces.
The default scorer/proposal distribution is the prior, which is roughly how Metaprob elevates a "gen" into an "inf".
Based on discussion with @alex-lew (please weigh in if there are errors in this summary), it seems sensible to allow the user to write a scorer that knows how to compute only some combinations of target and intervention traces and ask Metaprob to revert to the prior to handle the rest. This case arises when a user wants to optimize for a particular conditional query that they know how to handle well, but does not care about obtaining good estimates on other queries.
The proposition is thus to write a "scorer" generator that takes in a partial scorer and fills in the rest. There needs to be some thought for how (i) how to design the interface (for example, the user can specify a list of conditionals that they know how to propose); and (ii) the algorithms needed to complete the user-specified sketch (by reverting to the prior) in a probabilistically coherent way.
Marking as "research" which could serve as a good starter project.