search

salesforce / LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence
BSD 3-Clause "New" or "Revised" License
9.9k stars 971 forks source link

Confidence Score #293

Open shengyi4 opened 1 year ago

shengyi4 commented 1 year ago

How can i use BLIP ITM on text and image to predict confidence score? Any resource or notebook can you share? Is it possible to use lavis?

LiJunnan1992 commented 1 year ago

See https://github.com/salesforce/LAVIS/blob/main/examples/blip_image_text_matching.ipynb

Thanks.

shengyi4 commented 1 year ago

Thanks for your prompt reply! I trained the retrieval model on my own dataset, but here is the evaluate.txt output: {"txt_r1": 0.3333333333333333, "txt_r5": 0.6666666666666666, "txt_r10": 2.6666666666666665, "txt_r_mean": 1.222222222222222, "img_r1": 0.25170801869830994, "img_r5": 1.3807982740021576, "img_r10": 2.7400215749730314, "img_r_mean": 1.4575092892244996, "r_mean": 1.3398657557233609, "agg_metrics": 1.222222222222222} {"txt_r1": 0.3333333333333333, "txt_r5": 1.3333333333333333, "txt_r10": 2.6666666666666665, "txt_r_mean": 1.4444444444444444, "img_r1": 0.5037967289719626, "img_r5": 1.2704439252336448, "img_r10": 2.818341121495327, "img_r_mean": 1.5308605919003113, "r_mean": 1.4876525181723779, "agg_metrics": 1.4444444444444444} {"txt_r1": 0.0, "txt_r5": 1.6666666666666667, "txt_r10": 2.0, "txt_r_mean": 1.2222222222222223, "img_r1": 0.2732829917295937, "img_r5": 1.3448399856166846, "img_r10": 3.459187342682488, "img_r_mean": 1.6924367733429222, "r_mean": 1.4573294977825721, "agg_metrics": 1.2222222222222223} {"txt_r1": 0.3333333333333333, "txt_r5": 1.6666666666666667, "txt_r10": 2.0, "txt_r_mean": 1.3333333333333333, "img_r1": 0.4745911214953271, "img_r5": 1.7012266355140186, "img_r10": 2.8986565420560746, "img_r_mean": 1.6914914330218067, "r_mean": 1.51241238317757, "agg_metrics": 1.3333333333333333} {"txt_r1": 0.0, "txt_r5": 1.0, "txt_r10": 2.0, "txt_r_mean": 1.0, "img_r1": 0.30204962243797195, "img_r5": 1.524631427544049, "img_r10": 3.25782092772384, "img_r_mean": 1.6948339925686202, "r_mean": 1.34741699628431, "agg_metrics": 1.0} Is there a way to interpret the scores (if it is a good score)?