hao to Inference self datasts

[Danbinabo]

Hi！Solved the env problem,i want to Inference self dataset, you code used cocodatas, but in other data hao to test, example(img，segmention label png), when extract the language embeddings is in coco Clases, Will it affect use on other datasets, Can you explain in detail how to test inference on other datasets. thank you

[Danbinabo]

Can the data set be made into sam format,, i see you code "config/base/datasets/sam.py" can be used for test sam datasets?

[HarborYuan]

Hi,

It depends on the format of your data. I think you need to modify the dataset class, pipelines, and configs, which will be a great effort.

You can also refer to the demo (it is not hard to run on a local machine) if you only want to test several images. I recommend this plan since it will provide an interactive tool based on the web.

Please let me know if you have any other questions.

[Danbinabo]

example i have a self datasets,(img, png-label),i want to test seg miou, Can you provide an example or refine your project to support other data

[EricLabile]

Hi,

It depends on the format of your data. I think you need to modify the dataset class, pipelines, and configs, which will be a great effort.

You can also refer to the demo (it is not hard to run on a local machine) if you only want to test several images. I recommend this plan since it will provide an interactive tool based on the web.

Please let me know if you have any other questions.

What about inference on some standard datasets in the open-vocabulary segmentation like ADE-150, ADE-847, PC-59, PC-459? Is there any guidance on inference on these datasets? Thank you.

P.S. Why the paper doesn't test on the datasets mentioned above?

[HarborYuan]

Hi @EricLabile ,

Open-vocabulary SAM focuses on promptable instance segmentation. The datasets you mentioned are mainly focusing on semantic segmentation.

[EricLabile]

Hi @EricLabile ,

Open-vocabulary SAM focuses on promptable instance segmentation. The datasets you mentioned are mainly focusing on semantic segmentation.

Thank you for your response. Would you like to share some differences between promptable instance segmentation and semantic segmentation? Besides, could you provide a more detailed instruction on how to do the inference on datasets like ADE-150 and so on? Thank you very much!

[HarborYuan]

Hi @EricLabile @Danbinabo

Thanks for your interest in our work.

As I mentioned, semantic segmentation and promptable segmentation are two different settings. The first task needs the model to predict the semantic label for every pixel, while the second segment a single object based on the given prompt (bbox or point). That means, the goal of our open-vocabulary SAM is different from the open-vocabulary semantic segmentation methods.

I'm sorry that I'm currently working on other projects and may not be able to help you write code for additional datasets. My suggestion is to modify the dataloader according to the COCO dataset to adapt to your dataset. Alternatively, you can convert your own dataset into COCO JSON format.

BTW, it is not hard to adapt a dataset that is already in the mmdet. Please refer to mmdet to check whether the dataset you need is in it.

Feel free to let me know if you have any other questions.