search

dmmiller612 / bert-extractive-summarizer

Easy to use extractive text summarization with BERT
MIT License
1.38k stars 305 forks source link

Spacy and Transformers Dependency Upgrade #97

Closed igorbrigadir closed 3 years ago

igorbrigadir commented 3 years ago

Use spacy 3 model and sentencizer. This seems to be all that is needed to use Spacy 3. Thanks to https://github.com/dmmiller612/bert-extractive-summarizer/issues/95#issuecomment-775499397 pointers.

I also want to upgrade transformers version too but just going to double check it doesn't break anything.

igorbrigadir commented 3 years ago

https://github.com/huggingface/neuralcoref requires some pretty substantial changes to work with Spacy 3.0.0 https://github.com/huggingface/neuralcoref/issues/295 so it may be faster to either disable the coref part when using spacy 3.0.0 or replace it with an alternative.

dmmiller612 commented 3 years ago

yep, I recently made updates over the weekend in master to get it to work with both space 2 and 3. I'll update neuralcoref once they make the necessary changes.

igorbrigadir commented 3 years ago

Great!

Since I only needed a stopgap to not break my own pipeline, my solution was to rip it out entirely - so i'm gonna close this and update again when there's a new neuralcoref.