Closed zhiyuanyou closed 2 years ago
Hi. Yes, this can be regarded as "shared backbone" but we only share lower layers on ImageNet-LT. When having higher layers in experts, these experts would have different feature spaces, leading to more diverse experts.
Only classifiers in experts would sacrifice performance. However, I forget how much the decrease is, maybe 1 or 2 accuracy on ImageNet-LT (I am not very sure, maybe you can try).
Thanks for your response. I will have a try.
Hi~ Thanks for your excellent work. I have two questions about the paper and the code. (1) I notice that, in default, the shared backbone contains only the layer_1 & layer_2 of resent, other layers in resnet (layer_3 & layer_4) are all in the "expert". Could this setting be described as "shared backbone"? I mean, by saying "shared backbone", the readers will assume that only the classier layer is treated as the "expert". (2) Have you tried the setting that only the classier layer is treated as the "expert"? How much the performance decrease is?