search

thumbe3 / label-noise-nlp

11 stars 0 forks source link

Using BERT along your noise reduction model #3

Open sanazjb opened 3 years ago

sanazjb commented 3 years ago

Hello, thanks for the code. I was wondering if it is possible to use your code for noise reduction, whilst using BERT to train the classifier.

sid7954 commented 3 years ago

Hi!

Yes it should be possible to run our code for de-noising using the auxiliary noise model whilst using BERT for training the classifier. You will need to change the model class by instantiating a BERT-like model, which can be done using the Huggingface transformers repository. The track_training_loss function and BetaMixture1D class that implement the de-noising loss and fitting of the BMM just consider the predictions made by the model M which is defined in the text_pred class function. Please reach out if you have any further questions and let us know if this works for you.