search

bernard24 / RIS

Implementation of the approach described in the paper "Recurrent Instance Segmentation" https://arxiv.org/abs/1511.08250.
MIT License
27 stars 17 forks source link

How to implement crf-rnn in torch so it can be integrated in RIS model? #12

Closed brisker closed 7 years ago

brisker commented 7 years ago

How to implement crf-rnn in torch so it can be integrated in RIS model? https://github.com/torrvision/caffe/blob/e7f25fa2bdbb291a067930d60f337d305c128583/src/caffe/layers/multi_stage_meanfield.cpp

bernard24 commented 7 years ago

I am afraid that it is not straightforward. I am not aware of any implementation of crf-rnn on torch, so it would involve porting the original caffe code. It basically means to implement the bilateral filter using the permutohedral lattice to torch. Another option could be to call that code by means of lua interfaces, but I have 0 knowledge about how to do that. Maybe other people can help here?

brisker commented 7 years ago

@bernard24 besides, why do we need hungarian algorithm on ris model? Is this necessary? Just find the max IOU and assign the ground truth to the one with maximum IOU is not a good choice? For example, given an IOU matrix: (3 predictions and 2 ground truth) [ 0.1 0.2 0.5 0.2 0.4 0.3 ] Just assign prediction 1,2 3 with the ground truth 2 (0.2>0.1) , ground truth 2(0.4>0.2) and ground truth 1 (0.5>0.3) and it's done, right? Why bother to use hungarian algorithm?

bernard24 commented 7 years ago

Hi @brisker I am sorry I missed this question. The advantage of using the Hungarian algorithm for the loss is that if the segmentations made at first steps are mistaken (e.g. overlapping 2 or more ground truth segmentations) you still allow the model amend and make good predictions in that area in later iterations. In addition, Hungarian algorithm is not any bottleneck here, although it might be if the number of instances per image is higher than the ones in the dataset I used. Having said that, I did try the option you mentioned, and although results are not as good as when using the Hungarian loss, they are also decent.