fix #314 issue: add SEP_TOKEN at the end of tokenIDs when text_b does…

dbiir / UER-py

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

https://github.com/dbiir/UER-py/wiki

Apache License 2.0

3.01k stars 525 forks source link

Closed raytien closed 2 years ago

raytien commented 2 years ago

… not exist

raytien commented 2 years ago

fix #314 issue

ydli-ai commented 2 years ago

looks good to me