Why is cls_token_segment set to 1 and should it be set to 0?

ThilinaRajapakse / simpletransformers

Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI

https://simpletransformers.ai/

Apache License 2.0

4.12k stars 727 forks source link

Why is cls_token_segment set to 1 and should it be set to 0? #1514

Open peilongchencc opened 1 year ago

peilongchencc commented 1 year ago

https://github.com/ThilinaRajapakse/simpletransformers/blob/ffc829f18532d6920de22ee1c55c84ee1ce796d8/simpletransformers/ner/ner_utils.py#L422 Refer to line 442:cls_token_segment_id define the segment id associated to the CLS token (0 for BERT, 2 for XLNet)

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.