search

cnellington / Contextualized

An SKLearn-style toolbox for estimating and analyzing models, distributions, and functions with context-specific parameters.
http://contextualized.ml/
GNU General Public License v3.0
65 stars 9 forks source link

NOTMAD factor graphs #143

Closed blengerich closed 1 year ago

blengerich commented 1 year ago

To scale NOTMAD, it will be helpful to integrate factor graphs. This implementations creates a new attribute factor_mat_raw on initialization of the NOTMAD lightning module. The number of factors (latent dimensions) is governed by the parameter num_factors (values <= 0 or = X.shape[-1] disable factor graphs.

Presently, the tests verify that factor graphs train and converge to a solution that is better than initialization, but extensive testing and simulations are required to see how much they help in dimensionality reduction.

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

blengerich commented 1 year ago

This all looks great, I think there's two steps before merging:

  1. We should verify that factor graphs work as expected. I believe we've already run simulations to check that the low-dim factor approx does worse than full-dim when true_dims > factor_dims. We should also check that factor graphs do better than full dim graphs when the true graphs only have a few factors (true_dims == factor_dims).
  2. We should give a factors kwarg to the easy Bayesian networks.

There is now a demo notebook easy_bayesian_network_factors.ipynb which has a very basic investigation of the factor MSEs.

num_factors kwarg is now implemented for the easy Bayesian networks.