[lecture12][0521] Curious about residual network and inverted residual network

[koallako]

Regarding page 28 of yesterday's lecture slides,

On the left side, you mentioned that in the inverted residual network, the activation function is not used in the final 1x1 part due to the risk of information loss. So I thought the performance of this network should generally be better. However, on the right table, it shows that the performance of ReLU6 bottleneck is lower. I'm wondering if there is something wrong with my understanding on this part.

Minjung Kim (20210172)

[yjyoo3312]

@koallako Thank you for your comment! Yes, MobileNet v2 removes the activation function after the final 1x1 convolution, known as the linear bottleneck. The graph on the right compares two scenarios: one where the activation is removed (applying the linear bottleneck, shown in blue) and one where it is not removed (shown in green). The graph demonstrates that applying the linear bottleneck (removing the ReLU6 after the final 1x1 convolution) results in better accuracy. I believe your understanding is correct! :)

[koallako]

@yjyoo3312 Oh, I see. I was thinking linear bottleneck and Relu6 in bottleneck in reverse. Thank you for letting me know!!