search

language-brainscore / langbrainscore

[Marked for Deprecation. please visit https://github.com/brain-score/language for the migrated project] Benchmarking of Language Models using Human Neural and Behavioral experiment data
https://language-brainscore.github.io/langbrainscore/
MIT License
4 stars 1 forks source link

Feature tokenizer based span extraction 1 #32

Closed aalok-sathe closed 2 years ago

aalok-sathe commented 2 years ago

This code chunk provides a first-draft implementation of tokenizer-backed indices for extracting an appropriate subset of representations from reprs obtained using a larger context

aalok-sathe commented 2 years ago

currently not ready to merge:

Traceback (most recent call last):
  File "examples/test_mean_froi_pereira2018_firstsessions.py", line 132, in <module>
    main()
  File "examples/test_mean_froi_pereira2018_firstsessions.py", line 107, in main
    ann_modelcard = ann_enc.get_modelcard()
  File "/home/aalok/code/langbrainscore/langbrainscore/encoder/ann.py", line 271, in get_modelcard
    config_specs = {k: d_config[k] for k in config_specs_of_interest}
  File "/home/aalok/code/langbrainscore/langbrainscore/encoder/ann.py", line 271, in <dictcomp>
    config_specs = {k: d_config[k] for k in config_specs_of_interest}
KeyError: 'n_layer'

I think this is related to GPT vs BERT; will get back to this

aalok-sathe commented 2 years ago

#e066d6f solves this:

currently not ready to merge:

Traceback (most recent call last):
  File "examples/test_mean_froi_pereira2018_firstsessions.py", line 132, in <module>
    main()
  File "examples/test_mean_froi_pereira2018_firstsessions.py", line 107, in main
    ann_modelcard = ann_enc.get_modelcard()
  File "/home/aalok/code/langbrainscore/langbrainscore/encoder/ann.py", line 271, in get_modelcard
    config_specs = {k: d_config[k] for k in config_specs_of_interest}
  File "/home/aalok/code/langbrainscore/langbrainscore/encoder/ann.py", line 271, in <dictcomp>
    config_specs = {k: d_config[k] for k in config_specs_of_interest}
KeyError: 'n_layer'

I think this is related to GPT vs BERT; will get back to this