LAION-AI CLIP_benchmark issues

LAION-AI / CLIP_benchmark

CLIP-like model evaluation

MIT License

534 stars 68 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Webdatasets for Linear Probe evaluation

#127 shrebox opened 1 month ago
2
Questions about example for COCO captions captioning task

#126 strongweight opened 1 month ago
0
Leaking Validation Data in Few-Shot-Setting

#125 MarcoMorik opened 1 month ago
0
Duplicated `imagenet1k` classnames: "missile" & "sunglasses"

#124 EIFY opened 2 months ago
0
additional argument not recognized in MODEL_TYPES

#123 arnabk001 opened 3 months ago
0
update ImageNet V2 download urls

#122 shyammarjit opened 3 months ago
0
What is the composition of the dataset for model training

#121 geekchen007 opened 4 months ago
2
fix for missing sugar-crepe example

#120 samarth4149 closed 4 months ago
1
Sugar-CREPE eval swap_obj error

#119 samarth4149 closed 4 months ago
0
Move support of model types to all_clip module.

#118 rom1504 opened 5 months ago
2
Update babel image net

#117 rom1504 opened 5 months ago
0
Support winoground

#116 mehdidc closed 5 months ago
0
Add support of zero-shot classification/retrieval evaluation of captioning models using log-likelihood scoring, e.g. LLaVA, and others

#115 mehdidc opened 5 months ago
0
Fix crash when using `--distributed`

#114 bryant1410 closed 5 months ago
1
Multilingual benchmark datasets

#113 visheratin closed 7 months ago
9
Support computing metrics using test time distribution normalisation

#112 rom1504 opened 8 months ago
0
WDS classnames and empty lines

#111 mehdidc closed 7 months ago
2
Fix overwritten zeroshot templates issue #109.

#110 mehdidc closed 7 months ago
1
Overwritten zeroshot templates for webdataset eval

#109 djghosh13 closed 7 months ago
2
Benchmark siglip

#108 rom1504 closed 7 months ago
5
Improved linear evaluation that achieves better results

#107 teasgen closed 7 months ago
9
fix incorrect and missing splits

#106 mehdidc closed 10 months ago
0
support sharding evaluation based on rank, to enable parallel runs

#105 mehdidc closed 10 months ago
0
Incorrect and missed splits for some datasets

#104 teasgen closed 10 months ago
7
feed amp variable to zero_shot_classifier

#103 Separius closed 11 months ago
1
Support sugar crepe for compositionality evaluation

#102 mehdidc closed 11 months ago
1
add chinese flickr30k and flickr8k from https://github.com/li-xirong/cross-lingual-cap @yangapku

#101 mehdidc closed 11 months ago
3
support babel imagenet

#100 mehdidc closed 11 months ago
0
support both wds and coco-like datasets for captioning evaluation

#99 mehdidc closed 11 months ago
0
Custom classnames and templates

#98 mehdidc closed 11 months ago
0
Add compositionality benchmarks

#97 mehdidc opened 12 months ago
7
Add ImageNet-W

#96 mehdidc closed 11 months ago
1
Add babel imagenet

#95 rom1504 closed 11 months ago
4
Add additional tasks to ValueError

#94 moro-no-kimi closed 12 months ago
1
AttributeError: 'JaCLIPForBenchmark' object has no attribute 'eval'

#93 pkhk-1 opened 1 year ago
1
Catch len error in linear_probe

#92 djghosh13 closed 1 year ago
3
document how to run retrieval and generative eval using wds

#91 rom1504 closed 11 months ago
3
Performs normalization of the text embedding twice

#90 Adamdad closed 11 months ago
2
Release 1.4.0

#89 rom1504 closed 1 year ago
0
Fix silent webdataset error-handling

#88 djghosh13 closed 1 year ago
0
Error when test on multilingual_mscoco

#87 SingL3 closed 11 months ago
3
Fix issues #82, #83, #84, #85

#86 mehdidc closed 1 year ago
0
bug in "build_dataset" function

#85 JieyuZ2 closed 1 year ago
1
Please revise japanese clip's ReadMe and the position of model.eval in cli.py

#84 jackee777 closed 1 year ago
2
amp is always set to True

#83 SingL3 closed 1 year ago
1
Issue of model.txt in the readme example

#82 JieyuZ2 closed 1 year ago
1
Added wds/voc2007_multilabel

#81 djghosh13 closed 1 year ago
0
generate diagram as part of the cli

#80 rom1504 opened 1 year ago
1
Remove benchmark/ files and make it part of core

#79 rom1504 closed 11 months ago
1
Make verbose the default

#78 rom1504 closed 1 year ago
0