DFKI-NLP / sherlock

State-of-the-art Information Extraction
3 stars 1 forks source link

Train models on joint dataset and predict on Businesswire data #56

Open leonhardhennig opened 2 years ago

leonhardhennig commented 2 years ago

Train models on joint dataset of #54 and predict on Businesswire data (after NER validation by crowd workers)

phucdev commented 2 years ago

Until we get the NER validated Businesswire data we will use our RE model to predict on the Businesswire data with NER labels from our NER model ensemble (majority label vote), resulting in a weakly supervised dataset

phucdev commented 2 years ago

Our current best model that was trained on the joint data had the following evaluation results:

f1 = 0.8844128308597224 precision = 0.8759653163527977 recall = 0.8930248618784531

phucdev commented 2 years ago

The same model tested on the converted tacrev dataset achieved the following results:

f1 = 0.7601943798859075 precision = 0.7175907459114479 recall = 0.8081761006289309