Log determinant term off by constant for 5-bit images

openai / glow

Code for reproducing results in "Glow: Generative Flow with Invertible 1x1 Convolutions"

https://arxiv.org/abs/1807.03039

MIT License

3.11k stars 516 forks source link

Log determinant term off by constant for 5-bit images #51

Open lxuechen opened 6 years ago

lxuechen commented 6 years ago

Hi,

I understand that this might be a small issue and that you guys don't report bits/dim for celebA-HQ (256x256x3), but I think it's worth noting that the log-determinant might be off by a constant since we have an extra division in the preprocessing step (https://github.com/openai/glow/blob/654ddd0ddd976526824455074aa1eaaa92d095d8/model.py#L156).

If I'm not mistaken, this term is not added to the objective later on. Could you verify this? This might be of interest to future work which could intend on reporting the numbers for celebA-HQ (256x256x3)

Thanks, Chen

gerbenvv commented 5 years ago

The code in model.py essentially performs:

n_bins = 2^n_bits_x

x = floor(x / 2^(8 - n_bits_x))
x = x / n_bins - 0.5

Which can be more easily understood as:

n_bins = 2^n_bits_x

x = x / 2^8  # Scale.
x = floor(x * n_bins) / n_bins   # Zero n_bits_x least significant bits.
x = x - 0.5  # Center.

As the transformed inputs are modeled, no term is needed to account for these transformations.