search

brain-score / model-tools

Helper functions to extract model activations and translate from Machine Learning to Neuroscience
MIT License
8 stars 27 forks source link

key error when layers have different flattened coordinates #67

Open jenellefeather opened 1 year ago

jenellefeather commented 1 year ago

I was running brainscore on some transformer models and ran into an issue with channel_x not being a key in a dictionary. It seems to be noted in the code: https://github.com/brain-score/model-tools/blob/dfee128b4b5efea0cf0c29fd873de288d7c98153/model_tools/activations/core.py#L172

Log here for the failed model: http://braintree.mit.edu:8080/job/run_benchmarks/3861/parsed_console/job/run_benchmarks/3861/parsed_console/log.html

I found that I also could not include the final fc (logits) layer as one of the places where I was grabbing the activations, as this caused a key error ('embedding' was missing). I just removed these parts of the model from scoring, but it seems like something that others might run into.