brendanhasz / probflow

A Python package for building Bayesian models with TensorFlow or PyTorch
http://probflow.readthedocs.io
MIT License
171 stars 17 forks source link

Ordered Parameter #27

Open brendanhasz opened 4 years ago

brendanhasz commented 4 years ago

Add an OrderedParameter class where samples from the vector are always ordered (ie p = OrderedParameter(3); p[0] < p[1]; p[1] < p[2]).

This is trickier w/ SVI than w/ MCMC because you can't just do the exp/increment transform with independent variances, because the variances could cause some samples from adjacent parameters to be on the "wrong side" of each other.

Maybe do something like this? Where centered_vars make centered variables from raw ones using same QR transform as in https://github.com/brendanhasz/probflow/issues/19

def ordered_transform(vars):
    return vars[0] + tf.exp(vars[1]) * centered_vars(vars[2:])

Downside of that is each parameter's variance is correlated, but also variance depends on distance from mean?