uf-hobi-informatics-lab / ClinicalTransformerNER

a library for named entity recognition developed by UF HOBI NLP lab featuring SOTA algorithms
MIT License
145 stars 28 forks source link

continue train the NER model on new dataset #19

Closed bugface closed 2 years ago

bugface commented 3 years ago

currently, we do not have a train from where it left function. Every training starts from a new model (at least a new linear classification layer).

We need to implement continuing training function to support use cases like we want to train more epochs on the same data or train on new data with the exact same labels (no new labels are allowed)

bugface commented 3 years ago

we also need to create a mapping (BIO) from brat config during the data preprocessing instead from training data to cover all entity types (just in case some entity types defined but never used in one training data but appeared in other training data)