search

LutingWang / OADP

Object-Aware Distillation Pyramid for Open-Vocabulary Object Detection
Apache License 2.0
54 stars 3 forks source link

Difference between ml_coco.pth and vild.pth #8

Closed 455171668 closed 1 year ago

455171668 commented 1 year ago

What is the difference between ml_coco.pth and vild.pth? How to generate ml_coco.pth file? Looking forward to your reply

LutingWang commented 1 year ago

vild.pth is a file that contains the category embeddings used by ViLD, which uses an ensemble of multiple templates. Inspired by prompt tuning and DetPro, we use a prompt that is optimized using CLIP to perform multi-label classification on the MS-COCO dataset.

We are currently working on releasing the prompt-tuning script. We will also investigate the importance of ml_coco.pth.

455171668 commented 1 year ago

1、能否详细解释下ml_coco.pth 和 vild.pth内容上有什么区别，在您的代码里面没有看到ml_coco.pth文件的生成？

2、如果没有生成ml_coco.pth代码，能用文字详细描述下生成的逻辑么？

LutingWang commented 1 year ago

我们为所有类别名称的 embedding prepend 一个共享的 prompt，然后用 CLIP text encoder 提取类别特征。这个 prompt 是通过 prompt tuning 的方式，以图像多分类任务训练的。这部分训练代码还在整理，所以没有合入主分支。

1、能否详细解释下ml_coco.pth 和 vild.pth内容上有什么区别，在您的代码里面没有看到ml_coco.pth文件的生成？

2、如果没有生成ml_coco.pth代码，能用文字详细描述下生成的逻辑么？