deeppavlov / DeepPavlov

An open source library for deep learning end-to-end dialog systems and chatbots.
https://deeppavlov.ai
Apache License 2.0
6.67k stars 1.14k forks source link

How to train my data on Bert ranking ? #1081

Closed dimwael closed 4 years ago

dimwael commented 4 years ago

Please correct me if I am wrong, instead TF_IDF we can use Bert for classification like for the insuranceQA V1 ? (configs.rankings.ranking_insurance_interact model ) If yes, how can we train it on csv file ?

puleon commented 4 years ago

You can do it in the same way as with other ranking models. Please, see the documentation https://deeppavlov.readthedocs.io/en/master/features/models/neural_ranking.html#training-and-inference-on-your-own-data. That is to train on your own csv file you need to provide train, dev, test files in the appropriate file format and use the siamese_reader component as a reader and the siamese_iterator as an iterator in your configuration file.