search

NJU-LHRS / official-CMID

The official implementation of paper "Unified Self-Supervised Learning Framework for Remote Sensing Images".
80 stars 5 forks source link

关于自蒸馏与EMA的问题 #15

Closed XuanjiaZ closed 1 year ago

XuanjiaZ commented 1 year ago

你好 请问结构中的自蒸馏是只使用了EMA实现的吗 EMA实现的代码在哪里呢

XuanjiaZ commented 1 year ago

我好像已经找到了

pUmpKin-Co commented 1 year ago

Hi~您好. EMA相关代码是由/Pretrain/models/pretrain_model.py下line 76 MomentumUpdater以及hook(Pretrain\Exploring\hook\EMA_hook.py)一同完成的. 自蒸馏思想体现在EMA以及Loss的设计上.Loss可以参考/Pretrain/models/loss_utils.py下的LocalLoss.

XuanjiaZ commented 1 year ago

Hi~您好. EMA相关代码是由/Pretrain/models/pretrain_model.py下line 76 MomentumUpdater以及hook(Pretrain\Exploring\hook\EMA_hook.py)一同完成的. 自蒸馏思想体现在EMA以及Loss的设计上.Loss可以参考/Pretrain/models/loss_utils.py下的LocalLoss.

感谢!,已经找到了

pUmpKin-Co commented 1 year ago

Closed as long periods of inactivity, feel free to reopen if there is any problem.