search

ProsusAI / finBERT

Financial Sentiment Analysis with BERT
Apache License 2.0
1.42k stars 413 forks source link

Tokenizer #64

Closed pepi99 closed 2 years ago

pepi99 commented 2 years ago

Hello! Since FinBERT is further trained on BERT, and the original BERT has its own tokenizer, I suppose you are not training your own tokenizer to train the model, right? Otherwise, the model might get confused since the original model was trained with a different tokenizer.

doguaraci commented 2 years ago

Yes, we’re using BERT tokenizer.

pepi99 commented 2 years ago

This is very useful to know. Thank you and have a good day.