search

FoundationVision / GenerateU

[CVPR2024] Generative Region-Language Pretraining for Open-Ended Object Detection
130 stars 6 forks source link

COCO zero-shot #5

Open SCUTjinchengli opened 5 months ago

SCUTjinchengli commented 5 months ago

@clin1223 Hi, thanks for your significant work! We want to reproduce the COCO zero-shot results In Table 3. We generate the text embeddings via clip-vit-large-patch14-336. We replace the ZERO_SHOT_WEIGHT with the generated embeds. Unfortunately, the results are 0. Could you please give some points to us? Could you please provide the corresponding COCO-80-embeddings? Thanks! Have a nice day!

By the way, we generate the COCO-80-embeddings as follows.

model_path = "clip-vit-large-patch14-336"
model = CLIPTextModel.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
inputs = tokenizer(['a '+ class], padding=True, return_tensors="pt")
outputs = model(**inputs)
text_features = outputs.pooler_output

We obtain a numpy array, 80* 768.

hnanacc commented 2 months ago

+1, I am also getting poor results for datasets other than LVIS (0.1 - 5.0).