Named Entity Recognizer

[MaxAkbar]

Hello ML.NET,

Is there any way I can use ML.NET to created named entities?

Thanks, -Max

[Zruty0]

Currently, there is no component in ML.NET for named entity recognition. @GalOshri may be able to comment further with respect to future plans.

[Ivanidzo4ka]

Ping @GalOshri

[GalOshri]

We don't have immediate plans to add this right now, but it is on the backlog.

Does anyone have a specific scenario they are trying to enable and are blocked on this?

[MaxAkbar]

Hi Gal,

Yes, I am waiting on this and would love to have something I can use. I need to extract custom entities\Dates\Addresses\names and blocks of text from documents.

Let me know if you want a more detailed explanation.

I know this is on your backlog and can you let me know what version this is planned for?

-Max

[nimasTT]

Hi, i am using at the moment Stanford NLP (https://www.nuget.org/packages/Stanford.NLP.NER/) But it is just a Java Wrapper and doesn't support .Net Core. I would like to have more NLP (POS Tagger, NER, Named Entity Linking) possibilities native in C#.

[msamara]

Any update on this? Stanford's NER is not a viable option considering the lack of support of .NET Core

[tmarman]

+1

[rohittidke]

+1

[garywoodfine]

I would really like to see this functionality.

[ykafia]

Thinking of it, would it be probably a bonus to have a NLP premade tool (like spacy) for .NET in the future. When more NLP features will be added in the future, this would help for exploration.

[mayakfoury]

Please guys this is a very anticipated feature I would love to see, at the moment Stanford ner is the only decent library available and is not an option since it's heavily dependant on Java, either way, it has no support for .net core now.

[brykneval]

Plus Standford NLP is good for personal use and has commercial licence and usually scale and recognition is at commercial use

[codemzs]

@gvashishtha to drive this.

[ykafia]

Just an idea out of the box : With the coming of TorchSharp in ML.NET we could build a library upon different models like Alberta or GPT-2. We would only need an api around them to use in production.

[gvashishtha]

Hi all, I just joined the ML.NET team as a PM. I would appreciate understanding more about a) what scenarios you are trying to enable with Named Entity Recognition (NER) and b) what the impact of an ML.NET Named Entity Recognizer would be on your solution/business.

I notice that Stanford's NER primarily supports three classes: (PERSON, ORGANIZATION, LOCATION). Is this sufficient for all use cases?

[MaxAkbar]

Hello @gvashishtha, Standford NER model you were looking at was was probably trained on three entities. Go to this link and down to Model and notice that, based on the model, there are several more entities. If you look at their test server and click on the classifier, you will notice that it will have more entities. You can also get more info from here, and if you follow other links from that page, you can get to a better API sample.

Azure does NER pretty well, but the problem with Azure not to mention the cost :) is there is a limit to the amount of text you can send.

I think what would be best is to allow the API to accept text with annotation. The annotation would describe the entity type, so it should not be static.

I hope this helps.

[Edit] Found this article that allows you to create custom-named entities/

[codemzs]

@MaxAkbar You got it!

[gvashishtha]

Just to be clear, @MaxAkbar, when you say "Azure does NER," do you mean the Text Analytics API? https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-entity-linking

Additionally, can you confirm for me which of the Stanford capabilities you need for your application: 3 class, 4 class, or 7 class?

Model type	Included labels
3 class:	Location, Person, Organization
4 class:	Location, Person, Organization, Misc
7 class:	Location, Person, Organization, Money, Percent, Date, Time

[MaxAkbar]

Hi @gvashishtha,

Sorry I was not clear. I was referring to LUIS. At the time when I was searching for NER, Azure didn't have a NER feature, or I didn't look hard enough, just LUIS. That was a long time ago, :).

Anyway, LUIS has a feature called Entities. You provide an utterance, then mark the word or words and then add a label to identify the entity.

For example:

In the image above, we are providing utterances then labeling them with a custom entity. I think internally having known entities like Location, Person, Organization, Money, Percent, Date, Time is fine, but there should also be a feature to add custom entities.

[Edit] Forgot to note that my application I need to extract names but they must not be labeled name. For example, I need the name of the insurer vs. the name of the insured or seller vs. buyer.

I hope this helps. Max

[njfm0001]

Hi @gvashishtha,

I would love to see a functioning C# NER library that lets you train your own model with feature engineering, custom categories, and user-friendly parameterization. I found the RNNSharp library very helpful for NER development in C#. You might benefit from having a look at it. If I am not mistaken, it makes use of neural networks (bidirectional LSTM) for sequence labeling tasks such as NER.

Hope you can find that of use. Nicolás

[gvashishtha]

@MaxAkbar @njfm0001 have you looked into this library? https://github.com/microsoft/Recognizers-Text/tree/master/.NET

[njfm0001]

@gvashishtha As far as I can see, that library doesn't support PERSON, LOCATION or ORGANIZATION types, but dates, numbers, emails...

[MaxAkbar]

Hello @gvashishtha,

Thank you for providing the link to the text recognizers. I had looked at them when I was working with LUIS. I am using the recognizers in my current project.

The recognizers, in my opinion, is designed to extract written entities into numerical, date, and other formats. They identify a pattern and transform it, whereas NLP extracts entities based on grammar.

The underlying engine of the recognizers is regular expressions. For example, "I have two apples" when used in the recognizer will return the number 2, where I would identify the entities "I = Person" and "Apple = Fruit."

I hope this clarifies the requirements.

[derekantrican]

I would also like to see this. My scenario is that I want to recognize rock climbing related names & locations out of sentence. I have already "classified" some data like:

Bouldering in Central Park!!||Central Park
Not the best angle but check out that latch!!! Golden Bowl (V7) in Squamish||Golden Bowl||Squamish  
Does anyone have a used crash pad for sale?||

(where I have a sentence followed by || then all the names/locations separated again by ||)

[keithrowe]

Another vote for an ML.NET implementation of NER.

We have a commercial application that runs on the user's machine locally - no cloud processing yet. We would like to be able to do 7-class Named Entity extraction on large bodies of text.

[hobbsa]

+1 to NER Also, I think from a .NET perspective, something like spaCY would be the best use case. We use it now (because there is no .Net equivalent) and it works great.

1. Start with POS tagging so it then becomes easier to understand tokens in context.
2. Provide ability to train custom tokens (a great example out there is the Go/Golang training videos on using spaCY)
3. Provide out of box, pre-trained models for people, places, etc.

MS Video Indexer seems to have a great implementation of this for indexing videos and understanding topics, words, expressions, etc.

[moria97]

+1 to NER. We are trying to recognize personal informations from our train data, including

• AGE
• CREDIT_CARD_NUMBER
• DATE
• EMAIL_ADDRESS
• IP_ADDRESS
• LOCATION
• MAC_ADDRESS
• PASSPORT
• PERSON_NAME
• PHONE_NUMBER
• SWIFT_CODE
• US_DRIVERS_LICENSE_NUMBER
• US_SOCIAL_SECURITY_NUMBER
• US_VEHICLE_IDENTIFICATION_NUMBER
• US_INDIVIDUAL_TAXPAYER_IDENTIFICATION_NUMBER

Looking forward to NER feature in ML.Net

[JaCraig]

+1 on this. I definitely need custom capabilities as I need to pull things like US district court information, trying to figure out who is the defendant and plaintiff, etc.

[chester89]

@gvashishtha can you provide some feedback? Are there plans to do this?

[kartikvega]

I agree with the above Comments. Here are the reasons to include.

1. NER can be used for .Net Core and UWP apps.
2. Stanford NLP uses IKVM which does not support .Net Core as of this time (only .Net framework), as an example this LOC from Stanford NLP will fail, var classifier = CRFClassifier.getClassifierNoExceptions( classifiersDirecrory + @"\english.all.3class.distsim.crf.ser.gz"); because lack of FileStream support.
3. IKVM END of Life and future support for Stanford NLP for .Net will be limited. So its highly unlikely a future version of Stanford NLP will support next framework or .Net Core releases. https://sergey-tihon.github.io/Stanford.NLP.NET/faq.html
4. Azure TextAnalytics is a good option but would at scale would be good for sending batches of text and wait times/user experience management will be a hassle to manage on ASP.Net application looking for real time NER.
5. With PII extraction being important would make sense to include NER in ML.Net.

So @gvashishtha any updates on timelines or plans to include this in any future release?

[gvashishtha]

Sorry folks, I've since moved teams and no longer work on ML.NET @natke to triage.

[AniaBerthelot]

Hi, do you have any updates about this request please?

Thank you for your efforts, and the good quality of your work

[truencoa]

+1 ML.NET NER

[AnQueth]

driver license NER please. Name, address, city, state, license number, etc... don't want to mark up where this text is positioned for each state's version that can change at any time.

[shahiddev]

The built in NER from Text analytics only gets us so far - it would be awesome to be able to use ML .NET to either build or further train a model based on that capability so it can recognise entities in the context of our domain

[PaulDMendoza]

Yes, I also need this. We need to be able to train a NER model for our system to detect addresses and parse them into pieces.

[ajahangard]

Since ML.Net supports ONNX you can convert one of BERT models in Hugging Face to ONNX and use it for NER. I tried https://huggingface.co/HooshvareLab/bert-fa-zwnj-base-ner and following this tutorial https://ian.bebbs.co.uk/posts/Unoonnx and after a few try and error I managed to make it work.

[GeorgeS2019]

For users here (e.g. @natke) who are interested what @ajahangard described of BERT ONNX and the reference article UnoOnnx

Do join us with feedback how to achieve a .NET version of Netron (electron) to best visualize e.g. BERT Onnx for intuitive integration of ONNX into .NET in ways that have not been addressed by Netron

[a-gubskiy]

NER was included into ML.NET roadmap: https://github.com/dotnet/machinelearning/blob/main/ROADMAP.md#named-entity-recognition-ner

It's amazing!

[fercom]

+1 Also need this, I need to identify key words from user inputs in bot framework chatbots

[StanislavPrusac]

+1 ML.NET NER

[papyr]

Hello I am using the latest ML update, and I cannot get NER to work natively in ML.NET.

I tried to follow a couple of other suggests

1. https://sergey-tihon.github.io/Stanford.NLP.NET/faq.html - does not support .NET Core
2. Azure is not an option, we need it native on prem for various reasons besides pricing and security/compliance etc.
3. Tried ONNX to convert a BERT model for NER's but there much information to chase around, since its not straight forward, I gave up after not being able to figure out the options for NER4.

[michaelgsharp]

@luisquintanilla I think this is another scenario that we should consider for our TorchSharp integration work. Its a pretty popular idea. Can you take a look?

[breadnone]

Any news on this? Also having it natively working instead of azure is a plus for many portability reasons.

Edit: Here for BertOnnx sample https://github.com/ibebbs/BertOnnx

[swidz]

+1

[agonzalezm]

+1 any sample using existing NET model with ml.net?

[luisquintanilla]

Thanks for the discussion everyone. NER is a scenario we are actively working to bring to ML.NET as a high-level API powered by TorchSharp (similar to the Text Classification API we recently introduced).

As part of this work we want to ensure that you're able to have a smooth end-to-end workflow from data prep to training to inferencing. With that in mind, I have a few questions:

1. What does the data you're using to train look like?
2. What are you using to tag / label your data? (i.e. software, tools, processes).
3. Is there a specific format you're using for your training data? If so, can you provide a sample of it?

Your feedback on these is greatly appreciated!

[fercom]

@luisquintanilla Hi Luis, for me there are two important features: entities and intents, like in: "I want to flight to Paris" the intent may be "Travel" and the entity will be "Paris", currently we only do this on Luis.ai, the entity tagging can be done by clicking on the words and selecting the name of our entities (like "destination" in the travel example)

[rpenha]

@luisquintanilla, I disagree @fercom in one point: intents are not the main motivation of NER. The most importante feature is extract information from unstructured text and classify it into predefined categories.

In my particular case, the data will look like this (highlights are the entities that I need to classify). The entities could be labeled in more than one category.

The data format is just a plain text like this:

Nº 1020542-11.2021.8.26.0576 - Processo Digital - Recurso Inominado Cível - São José do Rio Preto - Recorrente: Valéria Berti Andaló - Recorrido: Romano Calil e Marques Alves Advogados Associados e outros - Recorrido: Flavio Marques Alves - Recorrida: Maristela Queiroz - Magistrado(a) Paulo Sergio Romero Vicente Rodrigues - Deram provimento ao recurso. V. U.  - - PETIÇÕES COM OFENSAS PESSOAIS GRATUITAS E DESNECESSÁRIAS, SEM NEXO COM AS TESES EM DEBATE JUDICIAL. ATO ILÍCITO CARACTERIZADO. ATUAÇÃO FORA DOS LIMITES DA IMUNIDADE. COMPENSAÇÃO POR DANOS MORAIS ARBITRADA EM 10 S.M., EQUIIVALENTES A R$ 12.120,00. CORREÇÃO MONETÁRIA DO ARBITRAMENTO. JUROS LEGAIS DO ATO ILÍCITO (DATA DA PRIMEIRA PETIÇÃO OFENSIVA), SÚMULAS 54 E 362, DO STJ. RECORRIDOS SOLIDÁRIOS. SENTENÇA REFORMADA. RECURSO PROVIDO. Para eventual interposição de recurso extraordinário, comprovar o recolhimento de R$ 223,79 na Guia de Recolhimento da União - GRU, do tipo ‘Cobrança’ - Ficha de Compensação, a ser emitida no sítio eletrônico do Supremo Tribunal Federal (http://www.stf.jus.br www.stf.jus.br) ou recolhimento na plataforma PAG Tesouro, nos termos das Resoluções nºs 733/2021 e 766/2022; e para recursos não digitais ou para os digitais que contenham mídias ou outros objetos que devam ser remetidos via malote, o valor referente a porte de remessa e retorno em guia FEDTJ, código 140-6, no Banco do Brasil S.A. ou internet, conforme tabela \”D\” da Resolução nº 606 do STF, de 23 de Janeiro de 2018 e Provimento nº 831/2004 do CSM. - Advs: Lincoln Falcochio (OAB: 377686/SP) - Wesler Augusto de Lima Pereira (OAB: 214225/SP) - Gisele Bozzani Calil (OAB: 87314/SP) - Flavio Marques Alves (OAB: 82120/SP) - Marco Antonio Scarpassa (OAB: 185311/SP) - 8º andar - sala 805

It would be nice if there is a tool like AWS SageMaker Named Entity Recognition Labeling Job Console to label the entities:

Thanks!

[fercom]

@luisquintanilla, I disagree @fercom in one point: intents are not the main motivation of NER. The most importante feature is extract information from unstructured text and classify it into predefined categories.

In my particular case, the data will look like this (highlights are the entities that I need to classify). The entities could be labeled in more than one category.

The data format is just a plain text like this:

Nº 1020542-11.2021.8.26.0576 - Processo Digital - Recurso Inominado Cível - São José do Rio Preto - Recorrente: Valéria Berti Andaló - Recorrido: Romano Calil e Marques Alves Advogados Associados e outros - Recorrido: Flavio Marques Alves - Recorrida: Maristela Queiroz - Magistrado(a) Paulo Sergio Romero Vicente Rodrigues - Deram provimento ao recurso. V. U.  - - PETIÇÕES COM OFENSAS PESSOAIS GRATUITAS E DESNECESSÁRIAS, SEM NEXO COM AS TESES EM DEBATE JUDICIAL. ATO ILÍCITO CARACTERIZADO. ATUAÇÃO FORA DOS LIMITES DA IMUNIDADE. COMPENSAÇÃO POR DANOS MORAIS ARBITRADA EM 10 S.M., EQUIIVALENTES A R$ 12.120,00. CORREÇÃO MONETÁRIA DO ARBITRAMENTO. JUROS LEGAIS DO ATO ILÍCITO (DATA DA PRIMEIRA PETIÇÃO OFENSIVA), SÚMULAS 54 E 362, DO STJ. RECORRIDOS SOLIDÁRIOS. SENTENÇA REFORMADA. RECURSO PROVIDO. Para eventual interposição de recurso extraordinário, comprovar o recolhimento de R$ 223,79 na Guia de Recolhimento da União - GRU, do tipo ‘Cobrança’ - Ficha de Compensação, a ser emitida no sítio eletrônico do Supremo Tribunal Federal (http://www.stf.jus.br www.stf.jus.br) ou recolhimento na plataforma PAG Tesouro, nos termos das Resoluções nºs 733/2021 e 766/2022; e para recursos não digitais ou para os digitais que contenham mídias ou outros objetos que devam ser remetidos via malote, o valor referente a porte de remessa e retorno em guia FEDTJ, código 140-6, no Banco do Brasil S.A. ou internet, conforme tabela \”D\” da Resolução nº 606 do STF, de 23 de Janeiro de 2018 e Provimento nº 831/2004 do CSM. - Advs: Lincoln Falcochio (OAB: 377686/SP) - Wesler Augusto de Lima Pereira (OAB: 214225/SP) - Gisele Bozzani Calil (OAB: 87314/SP) - Flavio Marques Alves (OAB: 82120/SP) - Marco Antonio Scarpassa (OAB: 185311/SP) - 8º andar - sala 805

It would be nice if there is a tool like AWS SageMaker Named Entity Recognition Labeling Job Console to label the entities:

Thanks!

@rpenha In my experience we need both, I agree that named entities are the most important feature but for our use cases intents are also important, at least for chatbot development, currently we do this with Microsofts Luis.ai service in that format. I don't know if it is relevant but we have experience with the development of at least 17 projects with this technology