question

[canxue-123]

why i can't train a model that have open-voc ablility . and RemoteCLIP model can't detect anything . please help me

[lizzy8587]

why i can't train a model that have open-voc ablility . and RemoteCLIP model can't detect anything . please help me

Hi~Could you please provide more details about the training?

[canxue-123]

just use your code you give python tools/train.py configs/visdrone_step1_base.py

python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model faster-rcnn

tools/test.py configs/visdrone_step1_base.py work_dirs/visdrone_step1_base/iter_80000.pth

And another question that ----- what's the purpose of the merge weight and the self-train ?

[canxue-123]

and the config file (12b-10k), the last line , i change it to { load_from = 'checkpoints/RemoteCLIP-RN50.pt' } . is that right? or maybe the {new_model3.pth } that i trained

[canxue-123]

can you leave your email?

[canxue-123]

and the test output is the mAP just have 21%, it can't detect the unseen class . i think there is something wrong

[lizzy8587]

just use your code you give python tools/train.py configs/visdrone_step1_base.py

python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model faster-rcnn

tools/test.py configs/visdrone_step1_base.py work_dirs/visdrone_step1_base/iter_80000.pth

And another question that ----- what's the purpose of the merge weight and the self-train ?

1. the command for merge_weights.py should be: (#3 ) python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model soft-teacher
2. and the last line in your config file: load_from = 'checkpoints/new_model3.pth'
3. then, self-training python tools/train.py configs/visdrone_step2_castdet_12b_10k.py

My email: daodao123@sjtu.edu.cn

[canxue-123]

ok，thanks，i got the mAP with 0.387

[canxue-123]

this output is by the step1 with openovc and got 80000.pth， then merge-weight with Remoteclip-rn50 to the newmodel.pth
next self-training with step2 config file and got 10000.pth test config is step2 config (12b-10k) output is mAP 0.387 how to make 0.40 ? in your paper (how the Novel to be pre-known)?

[Kkubuck]

ok，thanks，i got the mAP with 0.387

I also encounter zero-shot performance of 0 during testing, with an overall mAP of 0.24. How did you resolve this issue?

1. Train Step 1
2. During the merge, remove the state_dict and then merge
3. Provide the path to the merged model and train Step 2

[canxue-123]

this output is by the step1 with openovc and got 80000.pth， then merge-weight with Remoteclip-rn50 to the newmodel.pth next self-training with step2 config file and got 10000.pth test config is step2 config (12b-10k) output is mAP 0.387 how to make 0.40 ? in your paper (how the Novel to be pre-known)?

just use this way to train and test