Run unreproducible competition models on 1.0

brain-score / vision

A framework for evaluating models on their alignment to brain and behavioral measurements (50+ benchmarks)

http://brain-score.org

MIT License

119 stars 74 forks source link

Closed deirdre-k closed 4 months ago

deirdre-k commented 4 months ago

Run the models and then analyze why they are mismatched.

Then, based on findings, decide which score should stay in the database.

mike-ferguson commented 4 months ago

Models to score om 1.0:

[x] cvt_cvt-w24-384-in22k_finetuned-in1k_4 (website score:0.488, 2.0 Score: 0.477, 1.0 score: 0.488)
[x] effnetb1_cutmixpatch_augmix_robust32_avge4e7_manylayers_324x288: (website score:0.539, 2.0 Score: 0.560, 1.0 Score: 0.539)
[x] custom_model_cv_18_dagger_408 (website score: 0.561, 2.0 Score: 0.563, 1.0 Score: 0.561)
[x] ViT_L_32_imagenet1k (website score: 0.454, 2.0 Score: 0.5, 1.0 score: 0.454)