Open Atul997 opened 3 years ago
Facing the same issue
@gserapio - You might find this example from medspacy helpful https://github.com/medspacy/medspacy/blob/master/medspacy/sentence_splitting.py
@gserapio - You might find this example from medspacy helpful https://github.com/medspacy/medspacy/blob/master/medspacy/sentence_splitting.py
Thanks @alexhamiltonRN !
I have installed current spacy version 3. 1 and running the example with some modifications but it keeps throwing error of
ValueError: [E030] Sentence boundaries unset. You can add the 'sentencizer' component to the pipeline with:
nlp.add_pipe('sentencizer'). Alternatively, add the dependency parser or sentence recognizer, or set sentence boundaries by setting
doc[i].is_sent_start.
Below is the code that I am using- import spacy import pysbd from spacy.language import Language
@Language.component("sbd") def pysbd_sentence_boundaries(doc): seg = pysbd.Segmenter(language="en", clean=False, char_span=True) sents_char_spans = seg.segment(doc.text) char_spans = [doc.char_span(sent_span.start, sent_span.end, alignment_mode='contract') for sent_span in sents_char_spans] start_token_ids = [span[0].idx for span in char_spans if span is not None] for token in doc: token.is_sent_start = True if token.idx in start_token_ids else False return doc
if name == "main": text = "My name is Jonas E. Smith. Please turn to p.55." nlp = spacy.blank('en') doc = nlp(text)
add as a spacy pipeline