code corresponding to sampling process and text list generation during training

SHI-Labs / OneFormer

OneFormer: One Transformer to Rule Universal Image Segmentation, arxiv 2022 / CVPR 2023

https://praeclarumjj3.github.io/oneformer

MIT License

1.39k stars 129 forks source link

code corresponding to sampling process and text list generation during training #74

Closed yq790 closed 12 months ago

yq790 commented 1 year ago

Thank you for your work on OneFormer! I have been reading the paper as well as the code. In the paper, the current training task is decided by uniformly sampling from {panoptic, semantic, instance}, but I did not find the corresponding code for that. Besides, I could not find the code for deriving the semantic and instance labels from the corresponding panoptic annotations and generating text lists (i.e. a photo with a car). Could you please let me know how did you accomplish those? Thank you!

praeclarumjj3 commented 1 year ago

Hi, @yq790, thanks for your interest in OneFormer.

We randomly sample the task inside the corresponding DatasetMapper. the probabilities are defined inside the config files.

https://github.com/SHI-Labs/OneFormer/blob/56799ef9e02968af4c7793b30deabcbeec29ffc0/configs/ade20k/Base-ADE20K-UnifiedSegmentation.yaml#L55-L57

https://github.com/SHI-Labs/OneFormer/blob/56799ef9e02968af4c7793b30deabcbeec29ffc0/oneformer/data/dataset_mappers/oneformer_unified_dataset_mapper.py#L351-L366

praeclarumjj3 commented 12 months ago

Closing, feel free to reopen.