search

naver / shine

[CVPR'24 Highlight] SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection
Other
42 stars 2 forks source link

Inquiry on Modifying SHiNe Framework for Trainable Extensions #5

Closed zhongzee closed 1 week ago

zhongzee commented 1 week ago

Thank you for your outstanding work on the SHiNe framework. I have been greatly inspired by your research. Could you please provide guidance on how to modify the current framework to allow for further training of the base method (such as Detic), so that it can be trained for a specified epochs instead of remaining in a train-free state?

OatmealLiu commented 1 week ago

Hi! Thanks for your interests to our work.

My suggestion for modifying the current framework is the following steps:

  1. Modify the dataset registries in the data forlder of each method (detic, vldet, codet), such as the files in detic/data/datasets to create training splits for the dataset you wanna train with. You just need to modify it for one detector, and copy-paste it other detectors, since they all use the same Detectron2 paradigm.
  2. Downloading the pre-trained backbones used by each method-i.e., you can find the one for Detic here.
  3. Following the traning instructions provided by Detic here, VLDet here, and CoDet here.

I guess you won't meet too many difficulties because the code framework of SHiNe strictly follows the orginal implementation.

Good luck!

zhongzee commented 1 week ago

Thank you for your response and guidance. I have encountered an issue while running the script shine/plant_llm_syn_hrchy_tree.py. The error message is as follows: FileNotFoundError: [Errno 2] No such file or directory: 'openset_lvis_oid_llm_answers/cleaned_oid_lvis_gpt_hrchy_l1.json'

Thank you once again for your support!

OatmealLiu commented 1 week ago

The file is here shine/lvis_llm_answers/cleaned_lvis_gpt_hrchy_l1.json. I will fix the path in the code. Note that LVIS and COCO are not our focus in the paper because they do not have high quality hierarchy info. to use. Common objects' hierarchies usually have large ambiguity.

zhongzee commented 1 week ago

Thank you for your reply and reminder！