search

mit-han-lab / tinyengine

[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory
https://mcunet.mit.edu
MIT License
792 stars 130 forks source link

About SE Block in tinyengine codes. #67

Closed gunjupark closed 1 year ago

gunjupark commented 1 year ago

Hi, I have some questions about tinyengine's codes.

While looking through tinyengine's codes, I found code in TfliteConvertor.py that handles SE Block. The code includes a comment that reads as follows:

#         -> MEAN -> MEAN -> PWCONV -> PWCONV -> | ADD -> MUL ->     |
#  DWCONV                                        |            -> MUL |
#

here are my questions:

  1. Is it correct that SE Block means Squeeze & Excitation module?

  2. If so, does "ADD -> MUL -> MUL" refer to the h-swish activation function that replaces sigmoid?

Thank you

meenchen commented 1 year ago

Hi @gunjupark,

  1. Is it correct that SE Block means Squeeze & Excitation module?

Yes, MEAN -> MEAN is the pooling layer and PWCONV -> PWCONV is the two fc layers.

  1. If so, does "ADD -> MUL -> MUL" refer to the h-swish activation function that replaces sigmoid?

Yes, ADD -> MUL -> MUL is the activation function.

You can refer to this model: https://github.com/mit-han-lab/tinyengine/blob/main/assets/mcunetv2_large.tflite to check the architecture.

gunjupark commented 1 year ago

I really appreciate you responding quickly to my question :)

I had missed that tflite model was there. Thanks you.