torrvision / crfasrnn

This repository contains the source code for the semantic image segmentation method described in the ICCV 2015 paper: Conditional Random Fields as Recurrent Neural Networks. http://crfasrnn.torr.vision/
Other
1.34k stars 462 forks source link

Some questions about the source code don't match the paper written? #132

Closed machanic closed 7 years ago

machanic commented 7 years ago

1. where is weighting filter outputs step in source code? algo1 I notice that the paper said there is weighting filter outputs step , but the source code meanfield_iteration.cpp in line 155 directly omit this step?

2. the paper and the code written about 1x1 conv don't match? in the paper said that the compatibility transform there is 1x1 conv to sovle this weight linear combination: com meanfield_iteration.cpp in line 155~162 directly omit this 1x1 conv?

//--------------------------- Compatibility multiplication ----------------
  //Result from message passing needs to be multiplied with compatibility values.
  for (int n = 0; n < num_; ++n) {
    caffe_cpu_gemm<Dtype>(CblasNoTrans, CblasNoTrans, channels_, num_pixels_,
        channels_, (Dtype) 1., this->blobs_[2]->cpu_data(),
        message_passing_.cpu_data() + message_passing_.offset(n), (Dtype) 0.,
        pairwise_.mutable_cpu_data() + pairwise_.offset(n));
  }
  1. in meanfield_iteration.cpp line : 175: vector<bool> eltwise_propagate_down(2, true); why is 2 set here, because sum_topvec only have 2 element in it?

3. in meanfield_iteration.cpp line : 182: caffe_cpu_gemm<Dtype>(CblasNoTrans, CblasTrans, channels_, channels_, num_pixels_, (Dtype) 1., pairwise_.cpu_diff() + pairwise_.offset(n), message_passing_.cpu_data() + message_passing_.offset(n), (Dtype) 1., this->blobs_[2]->mutable_cpu_diff()); why transpose messagepassing matrix here?

4. again in line 190 caffe_cpu_gemm<Dtype>(CblasTrans, CblasNoTrans, channels_, num_pixels_, channels_, (Dtype) 1., this->blobs_[2]->cpu_data(), pairwise_.cpu_diff() + pairwise_.offset(n), (Dtype) 0., message_passing_.mutable_cpu_diff() + message_passing_.offset(n)); why transpose this->blobs_[2] here?

bittnt commented 7 years ago

Hey, you should check both multi_stage_meanfield as well as the meanfield_iteration code.

sadeepj commented 7 years ago

Hi @sharpstill,

Unfortunately, we don't have enough bandwidth to answer this kind of queries in detail. But please note that there's no discrepancy between what's written in the paper and the implementation. Please find my short answers to your queries below:

1.) Line 143 - line 153 of meanfield_iteration.cpp does exactly this weighting.

2.) Line 155 does this. It's just a different way of implementing it, but the effect is the same.

3.) & 4.) Again, these computations agree with the steps described in the paper. Please refer to the documentation of the relevant Caffe functions and the BLAS API for help.