TidalPaladin / combustion

Collection of helpers for deep learning with PyTorch
Apache License 2.0
4 stars 1 forks source link

Incorrect output of `to_8bit` on non-float input #1

Closed TidalPaladin closed 4 years ago

TidalPaladin commented 4 years ago

Calling to_8bit on non-float input tensors produces incorrect results, likely due to how integer multiplication / division is handled. Likely fix is casting all inputs to float.

Code to replicate

x = torch.randint(0, 10000, (3, 10, 10))
output = to_8bit(x) # only 255 or 0