Open scarsty opened 1 year ago
Hello @scarsty the miopenConvolutionForwardBias
performs the common Conv + Bias operation.
y = x * w + b
where x
is the input, w
is the weights, b
is bias with as many elements as there are output channels and *
represents convolution. Recall that alpha
and beta
are restricted to 1
and 0
in MIOpen.
In light of the above, I am not sure I understand what you mean by the following:
y = (alpha + beta) * y + alpha * bias
This function only perform adding bias, no conv. The inputs have only y and b, no x.
@scarsty Do you still need assistance with this ticket? If not, please close the ticket. Thanks!
@scarsty Do you still need assistance with this ticket? If not, please close the ticket. Thanks!
I want to hear the reason to design this function like this from the miopen team
@scarsty Hi, the behavior of miopenConvolutionForwardBias is what you observed. As @JehandadKhan mentioned, when alpha = 1 and beta = 0, you have
y = y + bias
which satisfy the functions purpose:"Calculate element-wise scale and shift of a tensor via a bias tensor." https://rocm.docs.amd.com/projects/MIOpen/en/latest/doxygen/html/group__convolutions.html#ga8e445590bd57cd0038324943791a9b4a. If possible, can you provide more context about your question?
I have tested some cases and found the behavior of this function is:
This make me very confused. In cudnn, function "cudnnAddTensor" is used to add bias, and the behavior of that is easy to understand: