Closed MrRace closed 2 years ago
Hi,
Since all the tokenizations are done in the training script, no preprocessing is needed for Chinese corpora. You may want to choose a Chinese pre-trained model or a multilingual pre-trained model for initialization though.
@gaotianyu1350 In my own corpus, I find about 25% characters is OOV when use official Chinese BERT-base. Therefore these characters will be regarded as UNK
, any advice?
Maybe use another Chinese pretrained model, for exae, Roberta based ones
If train SimCSE on Chinese corpus, does it need text preprocessing for example remove punctuations or emojis? Thanks a lot~