Share loc seed file generated method details for weakly supervised segmentation? Or demo code?

[TyroneLi]

Thanks~

[yaoqi-zd]

@TyroneLi please check the "Usage" part in the README. you can use 'infer_cam.py' to generate seeds from pretrained SGAN model.

[TyroneLi]

@TyroneLi please check the "Usage" part in the README. you can use 'infer_cam.py' to generate seeds from pretrained SGAN model.

Ok, thanks~ I should ask clearer, do you have code for generating seeds from pretrained VGG16 or resnet? Just like https://github.com/speedinghzl/DSRG, because I cannot find any related code or details for this part, while using both cam feature map and saliency mask for generating seed file. Thanks

[yaoqi-zd]

@TyroneLi actually generating seeds from VGG16/resnet or SGAN network are the same: 1) compute cam from vgg16/resnet or SGAN to get foreground seeds. you can just refer 'infer_cam.py' or any other WSSS repo for details; 2) compute background mask to get background seeds. you can use saliency as prior or just choose the low response region in cam itself as in AffinityNet's paper. 3) combine foreground and background seeds together, this part may differ in various WSSS papers and is a little tricky.

[TyroneLi]

@TyroneLi actually generating seeds from VGG16/resnet or SGAN network are the same:

1. compute cam from vgg16/resnet or SGAN to get foreground seeds. you can just refer 'infer_cam.py' or any other WSSS repo for details;
2. compute background mask to get background seeds. you can use saliency as prior or just choose the low response region in cam itself as in AffinityNet's paper.
3. combine foreground and background seeds together, this part may differ in various WSSS papers and is a little tricky.

I do really expect authors could mention more about point 3 trickies. However I cannot find any more details from DSRG or SEC...papers or their released code for this part. I try to retrain modified vgg16 according to most WSSS paper on voc using multi-label classification. But my generated seed worked really bad. I feed my retrained vgg16 cam foreground seed (reamin 20% of max value) to DSRG, but I got a bad result finally compared to DSGR.

[yaoqi-zd]

@TyroneLi actually I have implemented two methods for combine foreground and background seeds in infer_cam.py

in line 117,

def get_localization_cues_sec(att_maps, saliency, im_label, cam_thresh):

and in line 154,

def get_localization_cues_dcsp(att_maps, saliency, im_label, bg_thresh):

btw, SEC has described how to generate localization cues in section 6.1 of their paper.

as for your bad segmentation result. I recommend you to check your seed quality first before training a segmentation network. In our paper, we find the precision of seeds has a big impact on the final segmentation performance.

[TyroneLi]

@TyroneLi actually I have implemented two methods for combine foreground and background seeds in infer_cam.py

in line 117,

def get_localization_cues_sec(att_maps, saliency, im_label, cam_thresh):

and in line 154,

def get_localization_cues_dcsp(att_maps, saliency, im_label, bg_thresh):

btw, SEC has described how to generate localization cues in section 6.1 of their paper.

as for your bad segmentation result. I recommend you to check your seed quality first before training a segmentation network. In our paper, we find the precision of seeds has a big impact on the final segmentation performance.

Yeap, it is. But I wonder how did you get init seed mask inside your released downloaded file? Could you provide more informations about this part or have you released this part code?

[yaoqi-zd]

@TyroneLi OK, I get your point. I didn't release this part of code. it corresponds to tools/stage1 in the repo. to get the init seed mask, you need to train a VGG16 multi-label classification network(baseline model in our paper), and set foreground threshold to 0.3.

we have listed the precision and recall of seeds for all model variants. you can refer them to see if you get the right seeds. the init seed mask should have a precision higher than 80%.

[TyroneLi]

@TyroneLi OK, I get your point. I didn't release this part of code. it corresponds to tools/stage1 in the repo. to get the init seed mask, you need to train a VGG16 multi-label classification network(baseline model in our paper), and set foreground threshold to 0.3.

we have listed the precision and recall of seeds for all model variants. you can refer them to see if you get the right seeds. the init seed mask should have a precision higher than 80%.

Yeap, haha, why didn't u release this part code? Would u plan to release this part?hhh~

[yaoqi-zd]

@TyroneLi since it's only a trival vgg16 classification network and you can just remove the attention module and seed segmentation branch in our SGAN to implement it.

for initial seed generation, you may need to pay more attention to the seed computation part, rather than the classification network itself.

currently I'm already graduated from school and cannot access to the lab computer. sorry I may not be able to release this part of code. hope you can get init seeds with good quality.