search

ezyang / pytorch-unattached

Tensors and Dynamic neural networks in Python with strong GPU acceleration
http://pytorch.org
Other
20 stars 8 forks source link

forward() doesn't support non-Variable inputs #246

Open ezyang opened 7 years ago

ezyang commented 7 years ago

I originally thought that no one would actually do this, but I was wrong! In bnlstm, https://github.com/pytorch/benchmark/blob/master/benchmarks/models/bnlstm.py an integer "time" is passed in to SeparatedBatchNorm1d, which is used to determine which running_mean and running_var is used.

If there are many other incidences of this, we may have to support this.

jekbradbury commented 7 years ago

If the PyTorch API lets someone do something, they'll do it... but yeah recurrent batchnorm is the only situation I know of where you basically can't refactor such an auxiliary input out