search

ROCm / MIOpen

AMD's Machine Intelligence Library
https://rocm.docs.amd.com/projects/MIOpen/en/latest/
Other
1.07k stars 224 forks source link

The behavior of miopenConvolutionForwardBias #2428

Open scarsty opened 1 year ago

scarsty commented 1 year ago

I have tested some cases and found the behavior of this function is:

y = (alpha + beta) * y + alpha * bias

This make me very confused. In cudnn, function "cudnnAddTensor" is used to add bias, and the behavior of that is easy to understand:

C = alpha * A + beta * C
JehandadKhan commented 1 year ago

Hello @scarsty the miopenConvolutionForwardBias performs the common Conv + Bias operation. 

y = x * w + b

where x is the input, w is the weights, b is bias with as many elements as there are output channels and * represents convolution. Recall that alpha and beta are restricted to 1 and 0 in MIOpen.

In light of the above, I am not sure I understand what you mean by the following:

y = (alpha + beta) * y + alpha * bias
scarsty commented 1 year ago

This function only perform adding bias, no conv. The inputs have only y and b, no x.

ppanchad-amd commented 4 months ago

@scarsty Do you still need assistance with this ticket? If not, please close the ticket. Thanks!

scarsty commented 3 months ago

@scarsty Do you still need assistance with this ticket? If not, please close the ticket. Thanks!

I want to hear the reason to design this function like this from the miopen team

huanrwan-amd commented 1 week ago

@scarsty Hi, the behavior of miopenConvolutionForwardBias is what you observed. As @JehandadKhan mentioned, when alpha = 1 and beta = 0, you have y = y + bias which satisfy the functions purpose:"Calculate element-wise scale and shift of a tensor via a bias tensor." https://rocm.docs.amd.com/projects/MIOpen/en/latest/doxygen/html/group__convolutions.html#ga8e445590bd57cd0038324943791a9b4a. If possible, can you provide more context about your question?