spaCy pipeline object for negating concepts in text. Based on the NegEx algorithm.
NegEx - A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries Chapman, Bridewell, Hanbury, Cooper, Buchanan https://doi.org/10.1006/jbin.2001.1029
Version 1.0 is a major version update providing support for spaCy 3.0's new interface for adding pipeline components. As a result, it is not backwards compatible with previous versions of negspacy.
If your project uses spaCy 2.3.5 or earlier, you will need to use version 0.1.9. See archived readme.
Install the library.
pip install negspacy
Import library and spaCy.
import spacy
from negspacy.negation import Negex
Load spacy language model. Add negspacy pipeline object. Filtering on entity types is optional.
nlp = spacy.load("en_core_web_sm")
nlp.add_pipe("negex", config={"ent_types":["PERSON","ORG"]})
View negations.
doc = nlp("She does not like Steve Jobs but likes Apple products.")
for e in doc.ents:
print(e.text, e._.negex)
Steve Jobs True
Apple False
Consider pairing with scispacy to find UMLS concepts in text and process negations.
Designate termset to use, en_clinical
is used by default.
en
= phrases for general english language texten_clinical
DEFAULT = adds phrases specific to clinical domain to general englishen_clinical_sensitive
= adds additional phrases to help rule out historical and possibly irrelevant entitiesTo set:
from negspacy.negation import Negex
from negspacy.termsets import termset
ts = termset("en")
nlp = spacy.load("en_core_web_sm")
nlp.add_pipe(
"negex",
config={
"neg_termset":ts.get_patterns()
}
)
Replace all patterns with your own set
nlp = spacy.load("en_core_web_sm")
nlp.add_pipe(
"negex",
config={
"neg_termset":{
"pseudo_negations": ["might not"],
"preceding_negations": ["not"],
"following_negations":["declined"],
"termination": ["but","however"]
}
}
)
Add and remove individual patterns on the fly from built-in termsets
from negspacy.termsets import termset
ts = termset("en")
ts.add_patterns({
"pseudo_negations": ["my favorite pattern"],
"termination": ["these are", "great patterns", "but"],
"preceding_negations": ["wow a negation"],
"following_negations": ["extra negation"],
})
#OR
ts.remove_patterns(
{
"termination": ["these are", "great patterns"],
"pseudo_negations": ["my favorite pattern"],
"preceding_negations": ["denied", "wow a negation"],
"following_negations": ["unlikely", "extra negation"],
}
)
View patterns in use
from negspacy.termsets import termset
ts = termset("en_clinical")
print(ts.get_patterns())
Depending on the Named Entity Recognition model you are using, you may have negations "chunked together" with nouns. For example:
nlp = spacy.load("en_core_sci_sm")
doc = nlp("There is no headache.")
for e in doc.ents:
print(e.text)
# no headache
This would cause the Negex algorithm to miss the preceding negation. To account for this, you can add a chunk_prefix
:
nlp = spacy.load("en_core_sci_sm")
ts = termset("en_clinical")
nlp.add_pipe(
"negex",
config={
"chunk_prefix": ["no"],
},
last=True,
)
doc = nlp("There is no headache.")
for e in doc.ents:
print(e.text, e._.negex)
# no headache True
This library is featured in the spaCy Universe. Check it out for other useful libraries and inspiration.
If you're looking for a spaCy pipeline object to extract values that correspond to a named entity (e.g., birth dates, account numbers, or laboratory results) take a look at extractacy.