search

ScalaConsultants / Aspect-Based-Sentiment-Analysis

💭 Aspect-Based-Sentiment-Analysis: Transformer & Explainable ML (TensorFlow)
Apache License 2.0
539 stars 91 forks source link

How to fine tune this model #67

Open pepi99 opened 2 years ago

pepi99 commented 2 years ago

Hello, I am wondering - is it possible to fine-tune this model for some domain/specific words? I have training data, but I am not sure how exactly to fine tune what is already trained.

My idea is to use transfer learning on the trained Dense layer and preserve the Bert main layer. Is there an intuitive way to do that?

Cheers, Petar

pepi99 commented 2 years ago

I thought about this, and this approach might lead to catastrophic forgetting. What if I just add some new examples into the SemEval task 4 dataset and retrain the model ?