Open jamesjin0516 opened 4 months ago
Single-class sequence classification with BERT and RoBERTa now allows primary attributions. tox command run.
tox
LM class: new classify method. Mimics generate in tokenization, forward pass, and hidden states collection.
LM
classify
generate
attributions.py: model_forward's softmax step also allows 2D model outputs https://github.com/jamesjin0516/ecco/blob/558cb1e9d35c2551e1b4ad3add497124b0a48919/src/ecco/attribution.py#L68
model_forward
OutputSeq class: all_token_ids produced by classify currently contains index of predicted label, incorrectly decoded as a token.
OutputSeq
all_token_ids
Potentially include support for multi-class classification
64 @jalammar
Single-class sequence classification with BERT and RoBERTa now allows primary attributions.
toxcommand run.Changes
LMclass: newclassifymethod. Mimicsgeneratein tokenization, forward pass, and hidden states collection.attributions.py:
model_forward's softmax step also allows 2D model outputs https://github.com/jamesjin0516/ecco/blob/558cb1e9d35c2551e1b4ad3add497124b0a48919/src/ecco/attribution.py#L68TODOs
OutputSeqclass:all_token_idsproduced byclassifycurrently contains index of predicted label, incorrectly decoded as a token.Potentially include support for multi-class classification