Possible bug in IM2COL kernel

ehw-fit / tf-approximate

Approximate layers - TensorFlow extension

MIT License

26 stars 10 forks source link

Possible bug in IM2COL kernel #7

Closed AaronJing closed 4 years ago

AaronJing commented 4 years ago

https://github.com/ehw-fit/tf-approximate/blob/6a726b95a1f040230692228ec33e393668084676/tf2/src/cuda/approx_nn_conv_kernels.cu#L602 The following two lines should be signed, right?

FilipVaverka commented 4 years ago

You are right, that is indeed a bug. I think, it shouldn't be catastrophic as underflow in "inH" and "inW" should be caught by upper bound check. Either way, we will fix it with next update.