kanishkamisra / minicons

Utility for behavioral and representational analyses of Language Models
https://minicons.kanishka.website
MIT License
122 stars 29 forks source link

Add support for custom source sequences w/ Seq2SeqScorer #8

Closed aaronmueller closed 2 years ago

aaronmueller commented 2 years ago

Minor update: added support for custom source sequences when using Seq2SeqScorer.

Usage:

from minicons import scorer

s2s_scorer = Seq2SeqScorer("t5-base", "cpu")

source_seqs = ["form question: The man that hasn't seen the yaks has left.", "form question: The man that hasn't seen the yaks has left."]
target_seqs = ["Has the man that hasn't seen the yaks left?", "Hasn't the man that seen the yaks has left?"]

# Optional: use argument `source_format="custom"`
s2s_scorer.sequence_score(target_seqs, source=source_seqs)
netlify[bot] commented 2 years ago

Deploy Preview for pyminicons canceled.

Name Link
Latest commit 889e6b8989a16d27430042309169813d555012d2
Latest deploy log https://app.netlify.com/sites/pyminicons/deploys/62a5118204781c0008f8b480
kanishkamisra commented 2 years ago

Looks great! Thanks @aaronmueller!