helpmefindaname
commented
1 year ago Hi @kdk2612 You can do the following:
from flair.embeddings import TransformerEmbeddings
from flair.datasets import SENTEVAL_SST_GRANULAR, CONLL_03
from flair.models import TextClassifier, SequenceTagger
from flair.nn.multitask import make_multitask_model_and_corpus
from flair.trainers import ModelTrainer
# --- Embeddings that are shared by both models --- #
# use a transformer that can do both, sentence-embedding and word-embedding
shared_embedding = TransformerEmbeddings("distilbert-base-uncased", fine_tune=True, is_token_embedding=True, is_document_embedding=True)
# --- Task 1: Sentiment Analysis (5-class) --- #
corpus_1 = SENTEVAL_SST_GRANULAR()
model_1 = TextClassifier(shared_embedding,
label_dictionary=corpus_1.make_label_dictionary("class"),
label_type="class")
# -- Task 2: Named Entity Recognition on CONLL03 (english) -- #
corpus_2 = CONLL_03()
model_2 = SequenceTagger(shared_embedding,
label_dictionary=corpus_2.make_label_dictionary("ner"),
label_type="ner",
)
# -- Define mapping (which tagger should train on which model) -- #
multitask_model, multicorpus = make_multitask_model_and_corpus(
[
(model_1, corpus_1),
(model_2, corpus_2),
]
)
# -- Create model trainer and train -- #
trainer = ModelTrainer(multitask_model, multicorpus)
trainer.fine_tune(f"resources/taggers/multitask_test")If you have the annotations alligned, such that each sentence has Classification and NER labels, you can combine them:
def parse_annotations(my_data) -> Sentence:
# create a sentence with labels, might look different depending on your code
sentence = Sentence(my_data["text"])
sentence.add_label("class", my_data["label"])
for ner_label in my_data["ner"]:
sentence[ner_label["start"]: ner_label["end"]].add_label("ner", ner_label["label"])
# mark sentence for both tasks.
sentence.add_label("multitask_id", "Task_0")
sentence.add_label("multitask_id", "Task_1")
corpus = Corpus(train=[parse_annotations(annotation) for annotation in train_annotations], dev=...., test=...)
multitask_model = MultitaskModel([model_1, model_2], use_all_tasks=True) # `use_all_tasks` refers to training, ensuring that every sentence is trained on all tasks jointly. This is recommended if you use shared embeddings with alligned data, as it then will gather more training signal per batch (e.g. 4 sentences -> 8 annotations)
kdk2612
zrjohnnyl
Question
Hi, I am interested in doing Multi-task learning for NER and Sentence classification on a custom dataset. I was trying to find some tutorials or starting points but could not find any. Can you please provide some guidance on how to approach this?