Issue about high feature level set.

[Yangr116]

Hi, I have another question about your paper and codes.

In Eq. (8) of the paper, $If$ is used as the input of the level set energy loss, which should have the same meaning as the $I{img}$, right? However, in the code:

img_mst_tree = self.mst(img_target)
deep_stru_feature_img = self.tree_filter(mask_pred, img_target, img_mst_tree)

lst_mst_tree = self.mst(lst_target)
deep_stru_feature_lst = self.tree_filter(deep_stru_feature_img, lst_target, lst_mst_tree, low_tree=False)
high_feature = torch.cat((deep_stru_feature_img, deep_stru_feature_lst), dim=1) * box_mask_target

loss_feat_lst = self.loss_levelset(mask_scores_phi, high_feature, pixel_num) * 5.0

, high_feature is the refined mask score. How can I consider it a high-level feature? Did you try to use the feature level_set_feats in the level set loss directly?

[Yangr116]

May I see this format as the Tree Energy Loss? The difference is that the metric is changed to level set rather than L1 distance.

[LiWentomng]

@Yangr116 Hello, in the early version of this work exploration, I use the feature level_set_feats as the input data as the image for level set loss. It can also work for simple scene, like aerial scene and medical scene without the multiple objects and complex occlusion. The early exploration is here (technical report). This work can not work well for genecal coco.

For the complicate general coco, the direct level_set_feats can also work, but without the stable and advanced performance. To explore to obtain the better performance, I found the long-range operation can achieve the stable performance, like non-local and 'tree filter' operation. The tree_filter here is regarded as the structural feature enhance with long-range.

I also have tried the direct l1 loss in Tree Energy Loss，which achieves the performance drop 0.9-1.2 mAP, especially with large model as Swin-L based model.

[Yangr116]

I see. Thanks for your reply!