search

R1j1t / contextualSpellCheck

✔️Contextual word checker for better suggestions
MIT License
405 stars 56 forks source link

Pipeline dependencies? #61

Closed KennethEnevoldsen closed 2 years ago

KennethEnevoldsen commented 3 years ago

Wonderful project, seems a lot like something I would like to use for dacy, a danish NLP pipeline build on spacy. Currently, it seems like the spelling correction is dependent on the NER and other pipelines? If so this will sadly make it less useful for application before applying the pipeline.

R1j1t commented 3 years ago

Hi @KennethEnevoldsen It is great to hear you liked the project! NER and sentence tokenizer is an important to identify if the spelling is wrong of a named entity. Sentence tokenizer is required to provide context for spelling correction, so i am afraid I cannot do much at this point. I am occupied at present but once i get free I will surely have a look into dacy and ways I can modify contextualSpellCheck to make it more generalized.

KennethEnevoldsen commented 3 years ago

Sure thing, I naturally fully understand that you have other things. The reason I wanted to add a spelling correction module to the spacy pipeline to actually because relative modest spelling errors seem to influence the model performance notably. There is a more detailed explanation on this here.

R1j1t commented 3 years ago

I completely understand, will surely have try to contribute!!

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.