search

brain-score / vision

A framework for evaluating models on their alignment to brain and behavioral measurements (50+ benchmarks)
http://brain-score.org
MIT License
126 stars 79 forks source link

Score format maybe outdated in the tutorial and readme #562

Closed YudiXie closed 8 months ago

YudiXie commented 8 months ago

I found that the returned score format is perhaps outdated in this tutorial: https://www.brain-score.org/tutorial/quickstart

and the readme on the github page: https://github.com/brain-score/vision

xarray.Score (aggregation: 2)
array([0.01538053, 0.00571393])
Coordinates:
* aggregation  (aggregation) U6 'center' 'error'
Attributes:
raw:                   xarray.Score (aggregation: 2)>\narray([0.10192326...
ceiling:               xarray.Score (aggregation: 2)>\narray([8.21840601...
model_identifier:      pixels
benchmark_identifier:  dicarlo.MajajHong2015.IT-pls

In my recent usage, I believe the aggregation is removed and should be replaced by something like score.item() and score.error.item()?

mike-ferguson commented 8 months ago

Hey @YudiXie thanks for catching this! You are correct, and I will update it ASAP. This is also related to your other issue (#563), so I will address this very soon as well. Thanks again!