The role of feature disturbance.

[chmxu]

Hi, thanks for this impressive work. I have a question about the feature disturbance module. In my opinion, this process is like keeping erasing the most salient region in the support images whose saliency is decided according to the gradient. The less salient version of support images are provided in the training phase so that the trained model can be more robust in that the it can make use of all possible parts in target object. And I think this is proved by the super large improvement on APL of COCO. So how is this related to margin equilibrium?

[chmxu]

Also, how can I find the code for calculating mean prototypes after getting support features? It seems the model use a batch of train images as variable data to be the input of the model (train_decoupling_disturbance.py), which is then transformed into dynamic weight with network. But how can the prototypes $\mu_i$ of all base and novel classes be calculated from such a batch of data?

[Bohao-Lee]

Hi, thanks for this impressive work. I have a question about the feature disturbance module. In my opinion, this process is like keeping erasing the most salient region in the support images whose saliency is decided according to the gradient. The less salient version of support images are provided in the training phase so that the trained model can be more robust in that the it can make use of all possible parts in target object. And I think this is proved by the super large improvement on APL of COCO. So how is this related to margin equilibrium?

For the first problem, we hold the view that through feature disturbance, we can get more not quite similar novel data to train. Fig 5 of our paper, it shows that data from feature disturbance expanse the category boundary outward. And max margin loss shrink boundary. We implement class margin equilibrium in such an adversarial min-max fashion.

[Bohao-Lee]

Also, how can I find the code for calculating mean prototypes after getting support features? It seems the model use a batch of train images as variable data to be the input of the model (train_decoupling_disturbance.py), which is then transformed into dynamic weight with network. But how can the prototypes $\mu_i$ of all base and novel classes be calculated from such a batch of data?

We calculate the mean prototype in region_loss.py. In fact, you can change the output of network in darknet_decoupling.py.

[chmxu]

Also, how can I find the code for calculating mean prototypes after getting support features? It seems the model use a batch of train images as variable data to be the input of the model (train_decoupling_disturbance.py), which is then transformed into dynamic weight with network. But how can the prototypes $\mu_i$ of all base and novel classes be calculated from such a batch of data?

We calculate the mean prototype in region_loss.py. In fact, you can change the output of network in darknet_decoupling.py.

Thanks for your reply! As I can understand, the prototype in region_loss.py is represented by enews which is used to calculate max-margin loss in Eq.(3). But Fig.2 shows that the query features are activated by prototypes instead of the current dynamic weights which seem to be instance-level support features in each batch. I wonder if I misunderstand the framework or I miss anything in the code.

[Bohao-Lee]

Also, how can I find the code for calculating mean prototypes after getting support features? It seems the model use a batch of train images as variable data to be the input of the model (train_decoupling_disturbance.py), which is then transformed into dynamic weight with network. But how can the prototypes $\mu_i$ of all base and novel classes be calculated from such a batch of data?

We calculate the mean prototype in region_loss.py. In fact, you can change the output of network in darknet_decoupling.py.

Thanks for your reply! As I can understand, the prototype in region_loss.py is represented by enews which is used to calculate max-margin loss in Eq.(3). But Fig.2 shows that the query features are activated by prototypes instead of the current dynamic weights which seem to be instance-level support features in each batch. I wonder if I misunderstand the framework or I miss anything in the code.

I think you ignore the feature filtering. The prototype to count margin loss is filtered after feature filtering. We use the prototype which activates query feature to filter.

[chmxu]

Thank you!