Closed francois-rozet closed 2 years ago
@francois-rozet Thanks for bringing this up and explaining the design decisions! Looks like a very nice package. 👍 Happy to take a PR that adds lampe
to data/packages.yml
.
Thanks! I'll do it right away!
Done. By the way I see that Unconstrained Monotonic Neural Networks and Graphical Normalizing Flows are not referenced in the publication list. They are rather interesting!
Adding those would be great! And https://github.com/AWehenkel/UMNN under data/code.yml
too. Would you like to submit another PR?
@janosh I have submitted a PR with both papers.
Hello @janosh 👋 The normalizing flow implementations within the lampe
package have been exported to a standalone package called Zuko. I think the entry for lampe
should be replaced by zuko
. Should I send a PR to update this?
I think the entry for
lampe
should be replaced byzuko
. Should I send a PR to update this?
Yes, sounds good! Feel free to mention and link lampe
in the zuko
description as a downstream tool and what it's used for.
Hello :wave:,
I'm a researcher interested in simulation-based inference and posterior estimation. I have written a low-level library for amortized posterior estimation called lampe. Initially, LAMPE was relying on
nflows
for its normalizing flows, but it quickly became a limitation. I was not happy with some ofnflows
design choices. For instance, it is only possible to sample or evaluate batches and most operators do not support broadcasting. It is also not possible to use other networks than the built-in ones. I considered contributing tonflows
, but it seems the package is not actively developed anymore.So I decided to implement my own normalizing flows within LAMPE. The goal was to rely as much as possible onto the already existing distributions and transformations of PyTorch. Unfortunately, PyTorch distributions and transforms are not modules, meaning that they don't implement a
forward
method, you cannot send the parameters to GPU with.to('cuda')
or even get their parameters with.parameters()
. To solve this problem, LAMPE defines two (abstract) classes:DistributionModule
andTransformModule
. The former is anynn.Module
whoseforward
method returns a PyTorchDistribution
. Similarly, the latter is anynn.Module
whoseforward
method returns a PyTorchTransform
. Then, what is a normalizing flow? It is simply ann.Module
that is constructed from a baseDistributionModule
and a list ofTransformModule
.This design allows for very concise implementations of quite complex flows. Currently, LAMPE implements masked autoregressive flow (MAF), neural spline flow (NSF), neural autoregressive flow (NAF) and NAF based on unconstrained monotonic neural network (UMNN). All these flows support coupling (2 passes for inverse), fully autoregressive (as many passes as features) or anything in between (see Graphical Normalizing Flows). And all of that in about 800 lines of code, including whitespace and documentation. If you are interested, take a look at the transformations and flows.
Here is a small example with a neural spline flow (NSF).
The flow is currently a
nn.Module
. To condition the flow with respect to a contexty
, we call it. This returns a distribution which can be evaluated (log_prob
) or sampled (sample
) just like anytorch
distribution.