How you have appended Control Codes to the input, when training?

allenai / scitldr

https://scitldr.apps.allenai.org/

Apache License 2.0

746 stars 84 forks source link

How you have appended Control Codes to the input, when training? #4

Open shamanez opened 4 years ago

shamanez commented 4 years ago

Is it just by appending topic and TDLR relevant codes to the beginning of the text sequences? Do you use the usual GPT-Tokenizer on the Control Codes?