search

bayesiains / nflows

Normalizing flows in PyTorch
MIT License
851 stars 119 forks source link

allow negative scale in PointwiseAffineTransform #64

Closed mdmould closed 1 year ago

mdmould commented 2 years ago

The current implementation of PointwiseAffineTransform allows only positive values for the scale. As far as I can tell, this isn't necessary and only zero values (which make the transform no longer bijective) need to be rejected. Only the absolute value of the scale enters the log prob conversion while the scale and its reciprocal itself enters the transform of samples.

arturbekasov commented 2 years ago

Hey @mdmould (again),

Good idea. Left some comments on the implementation.

Cheers,

Artur

imurray commented 2 years ago

It's not clear to me that this change won't make some models fit worse than they did before. I'm not convinced it can be changed blindly (after all, if it made no difference, why bother?).

arturbekasov commented 2 years ago

It's not clear to me that this change won't make some models fit worse than they did before. I'm not convinced it can be changed blindly (after all, if it made no difference, why bother?).

@imurray This is a non-trainable transformation with fixed parameters (note both shift and scale are buffers), so this will have no impact on model fits. The only effect is that you can now apply e.g. PointwiseAffineTransform(shift=0, scale=-1) (y = -x).

mdmould commented 2 years ago

I renamed to _log_abs_scale and added test with negative scales.

Cheers, Matt

mdmould commented 2 years ago

I also added a test to check that scale=0 is caught.

arturbekasov commented 1 year ago

Thank you for taking the time, @mdmould, and sorry for the delay: merged!