fastText for NER

[PrashantRanjan09]

I read that using fastText embeddings can improve the NER. Can you please tell me how can we use fastText embeddings to get the NERs?

[loretoparisi]

@PrashantRanjan09 can you point us to the referred paper?

[sughodke]

It must be this one. https://arxiv.org/abs/1709.04820

[EdouardGrave]

Hi @PrashantRanjan09,

A standard approach to improve named entity recognition with continuous word representations is to use word embeddings, trained on large amount of unlabeled data, as feature in the NER system. Examples of this approach can be found in the following papers (in addition to the one from @sughodke's comment):

• Natural Language Processing (almost) from Scratch,
• Word representations: A simple and general method for semi-supervised learning.

This general idea was previously explored with discrete word representation (based on word clustering) instead of word embeddings:

• Name Tagging with Word Clusters and Discriminative Training,
• Semi-Supervised Learning for Natural Language.

Best, Edouard.

[DavidGOrtega]

I would recommend this paper also which is the foundation of polyglot ner

https://arxiv.org/pdf/1410.3791.pdf

[omerarshad]

any code available for https://arxiv.org/abs/1709.04820?