search

ZhangXu0963 / NPC

The code of the paper "Negative Pre-aware for Noisy Cross-modal Matching" in AAAI 2024.
12 stars 2 forks source link

about pretrained model #5

Open DongChenwei2000 opened 2 months ago

DongChenwei2000 commented 2 months ago

Hi, thank you for your excellent work. I have a question about reproducing your work. Have you used pre trained models before fine-tuning them? I know you have provided those models for evaluation, and by that, I mean whether pre trained models were used to partition noise based on loss values and whether NPC was trained.

ZhangXu0963 commented 2 months ago

We use the pre-trained model before fine-tuning it. But it's not used to partition the noise set. The idea in paper is not to pick out a noise set and perform some special processing on it. I'm not sure what's the meaning of "whether NPC was trained". Models we provided have been already fine-tuned by NPC(ours) or CLIP(baseline) on training dataset with different noise ratio. For example, npc_coco_20.pt is the model fine-tuned by NPC on MSCOCO training set with 20% noise. Hope you found this helpful.

DongChenwei2000 commented 2 months ago

Sorry, there may be some issues with my expression. What I mean is, did you load a pre trained model when training NPC? From your answer above, it seems that it has been loaded. Can you tell me where your pre trained model comes from? Anyway, this job is great. Thank you again.

DongChenwei2000 commented 2 months ago

We use the pre-trained model before fine-tuning it. But it's not used to partition the noise set. The idea in paper is not to pick out a noise set and perform some special processing on it. I'm not sure what's the meaning of "whether NPC was trained". Models we provided have been already fine-tuned by NPC(ours) or CLIP(baseline) on training dataset with different noise ratio. For example, npc_coco_20.pt is the model fine-tuned by NPC on MSCOCO training set with 20% noise. Hope you found this helpful.

Alright, I got it. When I actually ran it, I found that the program automatically downloaded the pre trained model. However, I also trained on a single RTX 3090 GPU, and encountered CUDA out of memory issues. Is there a problem with my settings? image

ZhangXu0963 commented 2 months ago

I'll show you the log file of training later. I need some time to re-run my code.