search

yikaiw / CEN

[TPAMI 2023, NeurIPS 2020] Code release for "Deep Multimodal Fusion by Channel Exchanging"
MIT License
284 stars 43 forks source link

About visualization figures in paper #2

Closed Sibyl233 closed 3 years ago

Sibyl233 commented 3 years ago

image Hi, thanks for your job. I wonder what 'averaged' means in Figure3, since the visualized feature maps are chosen by scaling factor in BN layer. And may I ask which layer/stage are these feature maps belonged to specifically？Because I really want to know whether the outdoor datasets have such characters. I'd be grateful if you would describe it in more detail.

yikaiw commented 3 years ago

In this work, feature maps for each layer can be categorized into three kinds: gamma_rgb>0, gamma_depth=0; gamma_rgb=0, gamma_depth>0; both>0. There won't be both=0 as we apply sparsity constraints on disjoint channels.

'averaged' means: 1. extracting feature maps at specific channels (e.g. channels with gamma_rgb>0, gamma_depth=0); 2. average these feature maps along the channels.

For plotting Figure 3, we use the 9th layer of ResNet, i.e. the 2nd layer of the 3rd stage (with 256 channels) of ResNet.

Sibyl233 commented 3 years ago

Thanks a lot for your reply 👍

cjs123456 commented 2 years ago

Thanks for your great job, but may I ask what the numbers in the legend (e.g. 0.0025, 0.0075) means? image It seems like they should be the values of scaling factors, but in the figure they should be the average of the chosen feature maps? In addition, did you add any filters to the average of the feature maps? They look very smooth.

yikaiw commented 2 years ago

Thanks for your great job, but may I ask what the numbers in the legend (e.g. 0.0025, 0.0075) means? image It seems like they should be the values of scaling factors, but in the figure they should be the average of the chosen feature maps? In addition, did you add any filters to the average of the feature maps? They look very smooth.

@cjs123456 They are the average of the chosen feature maps (after BN), and thus the values of scaling factors have been counted in. We do not apply filters on the averaged feature maps.

cjs123456 commented 2 years ago

Thanks for your great job, but may I ask what the numbers in the legend (e.g. 0.0025, 0.0075) means? image It seems like they should be the values of scaling factors, but in the figure they should be the average of the chosen feature maps? In addition, did you add any filters to the average of the feature maps? They look very smooth.

@cjs123456 They are the average of the chosen feature maps (after BN), and thus the values of scaling factors have been counted in. We do not apply filters on the averaged feature maps.

Thanks a lot for your reply. I know they are the average of the chosen feature maps，but what does the colorbar mean？A redder color in the figure means a higher scaling factor？or just a higher value of the averaged feature maps?

yikaiw commented 2 years ago

@cjs123456 A redder color means a higher value of the averaged feature maps.

cjs123456 commented 2 years ago

OK, thanks for your reply！