pytorch / android-demo-app

PyTorch android examples of usage in applications
1.47k stars 606 forks source link

integrate another model using a different tokenization method #273

Open BENSAFOUAN-Abdelhalim opened 1 year ago

BENSAFOUAN-Abdelhalim commented 1 year ago

I'm using the questionAnsewring App, but i want to change the model and integrate Camembert model, unfortunately they have different methods (DistilBERT used wordpiece but Camembert used SentencePiece), I can't find anything in the documentation on how to do this.