Flow models for 1-dimensional data

[Baukebrenninkmeijer]

I was looking at the modules and wasn't exactly clear on which flows are suitable for which purposes. I've tried most of them by now, but for example the NSF_CL and OneByOneConv do not work for 1-dimensional data.

The NSF_AR does work, but it only initializes the neural net from dim>1, so there's still very few parameters that it can train when it is 1-dimensional data, and thus the results are not that impressive in my testing.

Any thoughts in how you could use more parameters in the 1D setting?

[tonyduan]

Generally speaking, deep generative models (GANs, VAEs, Flows, Autoregressive) are only concerned with modeling p(x) where x is high-dimensional. The high dimensionality is what makes the problem challenging. If you're only interested in modeling p(x) for a scalar x you could simply, for example, use a [Gaussian kernel density estimate].

So as you pointed out, most flows (coupling layers, autoregressive layers, etc.) do not make sense in the 1D setting. That said, I'm vaguely aware of recent work [deep transformation models] that applies flow-like ideas to the univariate case.

[Baukebrenninkmeijer]

Thanks, I'll have a look. Maybe to explain a bit what I'm doing; I'm working on GANs for tabular data, and one of the main challenges is to model distributions of continuous columns in a good way. The current SOTA is using Gaussian mixture models for it, but those also have limited capacity for more complex distributions like those with long tails.

So my idea was to train flow models to transform these distributions to a Gaussian before being used as input for the GAN. Assuming the GAN is then better in modelling these distributions, we should get data that resembles the real data more, after using the flow models inverse to get the original continuous distributions. Do you have any thoughts on this?

[tonyduan]

I'm not very familiar with GANs. As far as I know both GANs and Flow models do something similar in that they fit p(x) by transforming samples z ~ N(0, I) into x = f(z). The difference is that for a normalizing flow f must be bijective whereas for a GAN f can be arbitrary, usually a neural net.

So it's not clear to me why you need to mix both flows and GANs. I would try each of these models individually first before mixing them. They both try to solve the same problem.