Special token treatment.

[prwoolley]

By default, the tokenizer adds special tokens to the "input_ids", specifically [CLS] at the beginning and [SEP] at the end of each token array. Was DNABERT-2 trained with these tokens present? If so, has the [CLS] token been used for finetuning, as an alternative to mean pooling?

Thanks for the model!

[Zhihan1996]

Sorry for the late reply!

Yes, DNABERT-2 is trained with these tokens present. And by default, it is fine-tuned with [CLS] token instead of mean pooling.