hassonlab / 247-encoding

Contains python scripts for performing encoding on 247 data.
0 stars 9 forks source link

Better method to load predictions #68

Closed VeritasJoker closed 1 month ago

VeritasJoker commented 1 year ago

When we run encoding with glove embeddings but only using words with predictions from gpt2 (correct/incorrect/top30%/bot30%), we need to load gpt2 datums. Now the code is very hackish. Need a better method

zkokaja commented 1 year ago

option 1: flag to force glove embeddings instead of currently loaded model. be sure to remove tokens/words without glove embeddings. option 2: get pickling to generate a glove pickle with language model predictions