search

huggingface / neuralcoref

✨Fast Coreference Resolution in spaCy with Neural Networks
https://huggingface.co/coref/
MIT License
2.84k stars 474 forks source link

compatibility with spaCy 3.0 ? #295

Open d5555 opened 3 years ago

d5555 commented 3 years ago

Is there going to be neuralcoref compatible with spaCy 3.0 ?

svlandeg commented 3 years ago

Yes! This is definitely on our road map :-) I'll actually pin this issue because I'm sure others are interested as well.

asterbini commented 3 years ago

Plus 1

asterbini commented 3 years ago

After some rough paches I am able to start neuralcoref with spacy 3 but I obviously get the error ''Trying to read a Model that was created with an incompatible version of Thinc'' Is there a way to convert the models to the new Thinc version used in Spacy 3?

svlandeg commented 3 years ago

@asterbini : Due to the many internal changes between Thinc 7 and 8, and spaCy 2 and 3, I'm not sure this is going to be feasible. See also this discussion at the spaCy forum.

We have now started working on a coref component native in spaCy v3 though. This won't be finished tomorrow, but we're making progress. We're going to reuse what we can from this library (with attribution, ofcourse) and make sure it trains and runs with native Thinc models :-)

asterbini commented 3 years ago

I was starting to look for a way to export the neuralcoref model in spacy2 as text, and try to load it in spacy3 :) From your words this seems to be useless, thanks for letting me know.

svlandeg commented 3 years ago

I'm not a 100% sure though. If you have a working PyTorch model, that might operate with Thinc's new shim layers, but you'll have to write a custom component to wrap it in spaCy...

asterbini commented 3 years ago

But why the neuralcoref models should depend on the spacy3 models implementation? From a cursory glance to the NeuralCoref annotator class it seems to me that it reads only properties of the doc tokens that have not changed from v2 to v3. Then unless thinc 8 is changed radically wrt 7, (and by mapping Relu to ReLu and Linear to Affine) perhaps there is a way to convert the binary data model to thinc 8. (and yet, as I am a newbie of thinc and neuralcoref an I am sure I could have missed something)

KTRosenberg commented 3 years ago

@asterbini : Due to the many internal changes between Thinc 7 and 8, and spaCy 2 and 3, I'm not sure this is going to be feasible. See also this discussion at the spaCy forum.

We have now started working on a coref component native in spaCy v3 though. This won't be finished tomorrow, but we're making progress. We're going to reuse what we can from this library (with attribution, ofcourse) and make sure it trains and runs with native Thinc models :-)

Out of curiosity since I'm trying to integrate Spacy and Neuralcoref into a time-sensitive project, is the move to Spacy 3 going to take on the order of a couple of weeks, a couple of months, or a year? I'm not familiar with the amount of work necessary to do something like this.

svlandeg commented 3 years ago

I'd estimate a month, a couple of months at most.

KTRosenberg commented 3 years ago

Thanks! In that case, I might downgrade temporarily.

d5555 commented 3 years ago

I'd estimate a month, a couple of months at most.

That's great news! I can't wait :)

ZhuoruLin commented 3 years ago

Wow. Can't wait to use neuralcoref with SpaCy V3!

jazzdup commented 3 years ago

looking forward to this! thankyou!

telmomenezes commented 3 years ago

I'm staying with spaCy 2 until this comes out. Can't wait! Thanks for the great work!

lauwauw commented 3 years ago

Also looking forward to this! Hoping to use this in my project with the en_core_web_trf pipeline, which is dependent on spaCy 3. Thanks indeed for the great work so far!

chinmaydas96 commented 3 years ago

This is an awesome project try on, waiting for the spacy 3.0 compartibility.

grigvardanyan commented 3 years ago

Hi @svlandeg, do you have approximate estimation when neuralcoref for spacy3 will be released ? Thanks in advance

Ar9av commented 3 years ago

Following

yutong-fei commented 3 years ago

folloming

seanswyi commented 3 years ago

Also following.

svlandeg commented 3 years ago

Hi all, as a quick update that I also posted in a related thread.

While we (Explosion) originally intended to update neuralcoref in collaboration with HuggingFace (and @thomwolf had been very helpful with that), we've recently revamped spaCy to 3.0 and it turned out that quite a significant rewrite was necessary to support a native Thinc model in neuralcoref. Meanwhile also, research on coreference resolution had progressed further.

So long story short, we started working on a built-in coreference module that will be part of the core spaCy library. You can follow its development here: https://github.com/explosion/spaCy/pull/7264. You'll see that we've been actively working on this. Most of the code is written, but it needs some more debugging and fine-tuning.

I can't speak for HuggingFace and what they intend to do with neuralcoref going forward and whether they're planning more updates, but either way this will always be a useful library for the older versions of spaCy :-)

KTRosenberg commented 3 years ago

Hi all, as a quick update that I also posted in a related thread.

While we (Explosion) originally intended to update neuralcoref in collaboration with HuggingFace (and @thomwolf had been very helpful with that), we've recently revamped spaCy to 3.0 and it turned out that quite a significant rewrite was necessary to support a native Thinc model in neuralcoref. Meanwhile also, research on coreference resolution had progressed further.

So long story short, we started working on a built-in coreference module that will be part of the core spaCy library. You can follow its development here: explosion/spaCy#7264. You'll see that we've been actively working on this. Most of the code is written, but it needs some more debugging and fine-tuning.

I can't speak for HuggingFace and what they intend to do with neuralcoref going forward and whether they're planning more updates, but either way this will always be a useful library for the older versions of spaCy :-)

That sounds good! Would a reasonable ETA be ~1 month for debugging and testing?

svlandeg commented 3 years ago

It's always difficult to provide these kind of estimates, as priorities sometimes shift, and we definitely want to "battle test" this new feature properly before we'll make it public.

dogberto commented 3 years ago

Awesome to hear this is happening! Thank you for putting the work into this. Please let us know when you think you have a release date :)

KTRosenberg commented 3 years ago

What is the status on this? Hopefully it's going well.

nilesh commented 3 years ago

Eagerly waiting for the core spacy3 integration. I can see that the pull request is out there and work in progress to merge. @KTRosenberg, you can track the progress here - https://github.com/explosion/spaCy/pull/7264

danyang-rainbow commented 2 years ago

Looking forward to see the spacy 3.0 or even 3.1 integration!✔

grigvardanyan commented 2 years ago

@svlandeg do you have any news :) ?

svlandeg commented 2 years ago

https://github.com/explosion/spaCy/pull/7264#issuecomment-986957673 ;-)

qhungngo commented 2 years ago

Looking forward to using it in SpaCy 3.0

polm commented 1 year ago

The new coreference component for spaCy 3 has been released.

fakerybakery commented 6 months ago

Hi @polm, personally, I think the neuralcoref library is much easier to use than the built-in coreference for SpaCy. Is it possible to make this library into a wrapper for the built-in coreference and preserve the same API? Thank you!