search

flairNLP / flair

A very simple framework for state-of-the-art Natural Language Processing (NLP)
https://flairnlp.github.io/flair/
Other
13.88k stars 2.1k forks source link

Support for Cross-View Training? #296

Closed stefan-it closed 4 years ago

stefan-it commented 5 years ago

Hi,

a few weeks before the BERT paper (which seems to got all the attention), the "Semi-Supervised Sequence Modeling with Cross-View Training" [1],[2] paper was published. It shows a nice semi-supervised learning algorithm using a mix of labeled/unlabeled data. Cross-View Training for Named Entity Recognition introduces auxiliary prediction modules (four modules for NER). These modules see a different views of the (unlabeled data) input and are trained to agree with the prediction made by labeled data.

I think this would be a great addition to the contextualized string embeddings and btw. there's one interesting prediction module, called "future" that works like a language model, but instead of predicting the next word it predicts which named entity class comes next.

What do you think?

[1] "Semi-Supervised Sequence Modeling with Cross-View Training", Kevin Clark, Minh-Thang Luong, Christopher D. Manning, Quoc V. Le [2] TensorFlow code

alanakbik commented 5 years ago

Hi @stefan-it this is a great idea and one we've been discussing internally as well. The combination of both approaches might be really powerful for NER (and mabye other tasks).

Any volunteers for adding this to Flair? :)

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.