wangzy22 / P2P

[NeurIPS 2022 Spotlight] P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting
https://p2p.ivg-research.xyz
MIT License
126 stars 9 forks source link

Visualization #3

Closed sheshap closed 1 year ago

sheshap commented 1 year ago

Hi,

Thank you for sharing the code. Can you please provide instructions for the visualization results?

Thanks

LavenderLA commented 1 year ago

Hi, thanks for your interest in our work!

You can visualize the colorful projected image via the img tensor in this line: https://github.com/wangzy22/P2P/blob/13169a3029e0706b9de4d20b98662181699fc7d3/models/layers/encoder.py#L106

auniquesun commented 1 year ago

Hi, thanks for your interest in our work!

You can visualize the colorful projected image via the img tensor in this line:

https://github.com/wangzy22/P2P/blob/13169a3029e0706b9de4d20b98662181699fc7d3/models/layers/encoder.py#L106

Hi, author, thanks for sharing the paper and code!

I run the code you mentioned when testing with config p2p_ConvNeXt-B-1k on ModelNet40, and save all img in a batch (64 images) with opencv-python successfully, as the code shows.

@staticmethod
def save_imgs(imgs):
    imgs = imgs.permute(0, 2, 3, 1)
    for i, img in enumerate(imgs):
        img = img.cpu().numpy()
        cv.imwrite(f'./img_{i}.png', img)

However, I found all saved images are just black background, an example is presented as below img_28

The results are obviously different from the projection visualization presented in the paper, what's the possible problem and can you give some insights or advice?

LavenderLA commented 1 year ago

Hi @auniquesun, thanks for your interest in our work!

We use to_pil_image function in torchvision to visualize our image. For example,

def vis_img(self, img):
    import torchvision.transforms.functional as tvF
    pic = tvF.to_pil_image(img)
    pic.save('vis_img.jpg')

Then you can use this function to visualize image:

img = self.img_layer(img_feat)  # B 3 224 224
self.vis_img(img[0].detach().cpu())
auniquesun commented 1 year ago

Thank you, I got it.