VCIP-RGBD / DFormer

[ICLR 2024] DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation
https://yinbow.github.io/Projects/DFormer/index.html
MIT License
142 stars 24 forks source link

Calculate the latency #13

Closed catchyoo closed 6 months ago

catchyoo commented 6 months ago

Hello.

Thank you for releasing good research.

When i check Appendix B(Details of DFormer), there are results of latency measurements.

Could you possibly share the code used for measuring latency?

yinbow commented 6 months ago

Thanks for your attention.

We upload the code for calculations of latency. You can run 'PYTHONPATH="$(dirname $0)/..":$PYTHONPATH python utils/latency.py --config local_configs.NYUDepthv2.DFormer_Large' to calculate the latency on your device. Note that the latency highly depends on the devices, including CPU and GPU. If you want to compare the latency, we recommend you to use the same device.

catchyoo commented 6 months ago

Thank you so much. And i have more questions.

1) Could you also share the code for visualizing the final segmentation map like Figure 14 in the paper? It's mentioned as visualization in infer.py, but it doesn't seem to be the case.

2) Could you also share the code for extracting features at stages as shown in Figure 10?

I'm very interested in this excellent paper, so I have a few requests. Thank you.

yinbow commented 6 months ago
  1. The color of different classes depends on the platte (this line). You can change the visualization color by modifying this line.

  2. You can transfer the feature in a randomly picked channel to a numpy array and visulize it to a gray image, such as:

    
      # the shape of x is [batch_size, channel, H, W]
      feature=x[j,i,:,:].view(x.shape[-2],x.shape[-1])
      feature=feature.cpu().numpy()
      feature-=feature.min()
      feature/=feature.max()+1e-10
      feature=np.round(feature*255)
      cv2.imwrite(dst_path,feature)


I hope this can help you.
catchyoo commented 6 months ago

Thank you