search

zhengye1995 / Zero-shot-Instance-Segmentation

code for CVPR paper Zero-shot Instance Segmentation
Apache License 2.0
93 stars 14 forks source link

How to Visualization single image or a directory of images? #14

Closed KainingYing closed 3 years ago

KainingYing commented 3 years ago

Mentioned above. Many thanks.

zhengye1995 commented 3 years ago

Using the visualization function in mmdetection: https://github.com/zhengye1995/Zero-shot-Instance-Segmentation/blob/6b25b318d7821b5f7a48f7d5e6dbd0bc63474e14/tools/test.py#L119

KainingYing commented 3 years ago

Thanks for your kindly reply. It seems can't save the processed image in the file.

KainingYing commented 3 years ago

Hi, I try to visualization the image with official weight. here is my command

python tools/test.py configs/zsi/48_17/test/zsi/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_decoder.py ./checkpoints/ZSI_48_17.pth --show

The results look not good in some image I random selected. Here they are. image image I also found the class of airplane is a ghost on the bus. I am confused whether I am wrong or this is the defect of ZSI. Thanks,

zhengye1995 commented 3 years ago

I think your visualization results are right and this is the problem of my work which need to be improved. Considering that the visual and semantic data we used are relatively simple, our mAP results in zero-shot setting are much lower than the results under supervised learning.

I think the biggest problem still comes from the dataset. For example, CLIP performs very well on zero-shot tasks when using a large number of visual-semantic data pairs to train its model. Using a large scale data with more effective semantic information can significantly improve the performance.

harrytrinh2 commented 3 years ago

@zhengye1995 @noobying How can I save the output images? I am working on a remote server, therefore, I cannot use --show

zhengye1995 commented 3 years ago

@zhengye1995 @noobying How can I save the output images? I am working on a remote server, therefore, I cannot use --show

See this code.

harrytrinh2 commented 3 years ago

How can I use that show_result function inside test.py?

zhengye1995 commented 3 years ago

How can I use that show_result function inside test.py?

The show_result is used here in test.py .

harrytrinh2 commented 3 years ago

@zhengye1995
I tried to do the inference with a single image only. Here is the code:


from mmdet.apis import init_detector, inference_detector, show_result
import mmcv

config_file = 'configs/zsi/48_17/test/zsi/zero-shot-mask-rcnn-BARPN-bbox_mask_sync_bg_decoder.py'
checkpoint_file = 'checkpoints/ZSI_48_17.pth'

# build the model from a config file and a checkpoint file
model = init_detector(config_file, checkpoint_file, device='cuda:0')

# test a single image and show the results
img = '000000581672.jpg'  # or img = mmcv.imread(img), which will only load it once
result = inference_detector(model, img)
# visualize the results in a new window
show_result(img, result, model.CLASSES)
# or save the visualization results to image files
show_result(img, result, model.CLASSES, out_file='000000581672_result.jpg')

And I got this error: missing keys in source state_dict: mask_head.con_vec_t.weight, mask_head.conv_vec_unseen.weight

The model and loaded state dict do not match exactly

missing keys in source state_dict: mask_head.con_vec_t.weight, mask_head.conv_vec_unseen.weight

Traceback (most recent call last):
  File "tools/load_checkpoint.py", line 18, in <module>
    model = init_detector(config_file, checkpoint_file, device='cuda:0')
  File "/home/harry/Documents/SERVERCODE/ZSI2_cu11/mmdet/apis/inference.py", line 37, in init_detector
    if 'CLASSES' in checkpoint['meta']:
KeyError: 'meta'