Closed amir-zeldes closed 4 years ago
@loganpeng1992 : this is an important architectural change which will allow us to incorporate document-level features in prediction. We'll do this in a couple of steps to make things less complicated. For now step 1:
No need to add any new functionality in step 1: just try to decouple reading the input for all sentences, so that we can call process_sentence on each of them later.
Another important thing: use the develop branch for this, not Chinese-dev
Implemented in V2.1
Refactor to read all sentences first, then send them incrementally to process_sentence in xrenner_xrenner.py. This will allow document level logic, such as genre classification on all tokens or doing a document length pass for entity recognition using CRF.