search

berkeley-hipie / HIPIE

[NeurIPS2023] Code release for "Hierarchical Open-vocabulary Universal Image Segmentation"
https://people.eecs.berkeley.edu/~xdwang/projects/HIPIE/
MIT License
260 stars 19 forks source link

Using different labels #8

Closed joshmyersdean closed 6 months ago

joshmyersdean commented 1 year ago

Hello!

Thank you for this great work and I am looking forward to the training code release. In the meantime, is there an easy way to provide my own labels to the demo? Specifically part labels.

Thank you!

KKallidromitis commented 1 year ago

Hi I have implemented a demo that uses a custom dataset (Demo-HIPIE-custom.ipynb). Please note "stuff" must always be after "things" and also that the current model uses the standard weights and config so for optimal parts results please change to the parts model/config.

joshmyersdean commented 1 year ago

Amazing, thank you!

joshmyersdean commented 11 months ago

Hi again!

For custom parts, could you explain how you constructed the dictionary items in get_args_eval()?

Thank you!

KKallidromitis commented 11 months ago

Hi, I am sorry for the delay. get_args_eval() is in the file projects/HIPIE/hipie/demo_lib/demo_utils.py it uses the MetadataCatalog from detectron2 and the functions get_openseg_labels,cat2ind_panoptics_coco from projects/hipie/data/coco_dataset_mapper_uni.py and projects/hipie/data/datasets/catids.py respectively.

For custom parts we dont explicitly call the function because it is used to load labels for existing datasets. For custom parts I imitate the format of labels in detectron by generating metadata using: make_custom_metadata. The function takes the custom labels as input and the metadata of coco_2017_train_panoptic_with_sem_seg for the color scheme.