Train Neuralcoref in new language

[csgomezg0]

Hi :), I want train Neuralcoref in language Spanish. First I try train Neuralcoref in English, I get a corpus with format *.v4_gold_conll from this url: https://github.com/clab/att-coref/tree/master/data/conll-2012 Then I saves this files in the folders train, test and dev.

With the scripts Conllparser.py and learn.py, I process the data and trained.

I get three files from this training:

• Jul25_22-52-53_95e13715414b_best_modelallpairs
• Jul25_22-52-53_95e13715414b_best_modelranking
• Jul25_22-52-53_95e13715414b_best_modeltoppairs

This files are weights and bias from the models trained.

I want use this models, but I don't know how. I try this, but I get error.:

import torch

tm = torch.load('./Jul25_22-52-53_95e13715414b_best_modelallpairs')

SIZE_SINGLE_IN=674
SIZE_PAIR_IN=1370
h1=1000
h2=h3=500

from thinc.v2v import Model, ReLu, Affine
from thinc.api import chain, clone

with Model.define_operators({'**': clone, '>>': chain}):
    single_model = ReLu(h1, SIZE_SINGLE_IN) >> ReLu(h2, h1) >> ReLu(h3, h2) >> Affine(1, h3) >> Affine(1, 1)
    pairs_model = ReLu(h1, SIZE_PAIR_IN) >> ReLu(h2, h1) >> ReLu(h3, h2) >> Affine(1, h3) >> Affine(1, 1)

for l in pairs_model._layers + single_model._layers:
        l.W
        l.b

pairs_model._layers[0].W = tm["pair_top.0.weight"].cpu().numpy()
pairs_model._layers[0].b = tm["pair_top.0.bias"].cpu().numpy()

pairs_model._layers[1].W = tm["pair_top.3.weight"].cpu().numpy()
pairs_model._layers[1].b = tm["pair_top.3.bias"].cpu().numpy()

pairs_model._layers[2].W = tm["pair_top.6.weight"].cpu().numpy()
pairs_model._layers[2].b = tm["pair_top.6.bias"].cpu().numpy()

pairs_model._layers[3].W = tm["pair_top.9.weight"].cpu().numpy()
pairs_model._layers[3].b = tm["pair_top.9.bias"].cpu().numpy()

pairs_model._layers[4].W = tm["pair_top.10.weight"].cpu().numpy()
pairs_model._layers[4].b = tm["pair_top.10.bias"].cpu().numpy()

single_model._layers[0].W = tm["single_top.0.weight"].cpu().numpy()
single_model._layers[0].b = tm["single_top.0.bias"].cpu().numpy()

single_model._layers[1].W = tm["single_top.3.weight"].cpu().numpy()
single_model._layers[1].b = tm["single_top.3.bias"].cpu().numpy()

single_model._layers[2].W = tm["single_top.6.weight"].cpu().numpy()
single_model._layers[2].b = tm["single_top.6.bias"].cpu().numpy()

single_model._layers[3].W = tm["single_top.9.weight"].cpu().numpy()
single_model._layers[3].b = tm["single_top.9.bias"].cpu().numpy()

single_model._layers[4].W = tm["single_top.10.weight"].cpu().numpy()
single_model._layers[4].b = tm["single_top.10.bias"].cpu().numpy()

import spacy
nlp = spacy.load("en_core_web_md")
print(nlp.vocab)
import neuralcoref
nc = neuralcoref.NeuralCoref(nlp.vocab)
nc.model = single_model, pairs_model
nlp.add_pipe(nc, "neuralcoref")
doc = nlp('My sister has a dog. She loves him.')

print('1 ',doc)
print('2',doc._.has_coref)
print('3',doc._.coref_clusters)

The error is:

  File "/usr/local/lib/python3.8/dist-packages/thinc/check.py", line 74, in has_shape_inner
    raise ShapeMismatchError(arg.shape, shape_values, shape)
thinc.exceptions.ShapeMismatchError:

  Shape mismatch: input (4, 668) not compatible with [None, 674].

  Traceback:
  ├─ __call__ in /usr/local/lib/python3.8/dist-packages/spacy/language.py:385
  ├─── __call__ in neural/_classes/model.py:167
  └───── predict in neural/_classes/feed_forward.py:40
         >>> X = layer(X)

Something it's wrong.

How I Can load this three models in spacy?

Note: spacy==2.1.3 ; neuralcoref=4.0 ; torch=1.7.1+cu101

[fernandoantoniodantas]

I'm also trying but for protugues.

[csgomezg0]

I'm also trying but for protugues.

Hi, @fernandoantoniodantas, I try with neuralcoref but it isn't work, then i am trying with this model, https://github.com/msg-systems/coreferee, if you get that neuralcoref work I would like to know how to get it.

If i can help you in something, I will be pending anything.

[AlanQuille]

Hello, I am having the exact same error. If anyone gets it working I would be very grateful.

[fernandoantoniodantas]

I'm also trying but for protugues.

Hi, @fernandoantoniodantas, I try with neuralcoref but it isn't work, then i am trying with this model, https://github.com/msg-systems/coreferee, if you get that neuralcoref work I would like to know how to get it.

If i can help you in something, I will be pending anything.

Did you try in portuguese?

[csgomezg0]

@fernandoantoniodantas. No, i don't try with portuguese, i try train neuralcoref with spanish.

[fernandoantoniodantas]

Do the corefee work with ConLL-U format?

Em qua., 1 de set. de 2021 10:11, csgomezg0 @.***> escreveu:

@fernandoantoniodantas https://github.com/fernandoantoniodantas. No, i don't try with portuguese, i try train neuralcoref with spanish.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/huggingface/neuralcoref/issues/321#issuecomment-910271439, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFB23ZM6MFESWGBS22PLWLT7YRBDANCNFSM5BA62DIA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[csgomezg0]

@fernandoantoniodantas. Coreferee use diferent format for default, format *.ann, but you try get data in format .ann for language portuguese or try transform format conll-U to .ann, also you can try change de scripts that load the data in the model of coreferee.

[hiiamsid]

@csgomezg0 i was also trying to build Spanish coreference resolution resolver and so far no success have been achieved so far. I had a look on coreferee but on adding new language it is required to have better understanding of grammar of that specific language unfortunately i lack that. Can you help me with this? Also did that model performed well?

[csgomezg0]

Hi @hiiamsid, this is my email csgomezg0@gmail.com, maybe we can colaborate, I get a model with coreferee but this model must be improved.

[ValerioDiCarloBUP]

@csgomezg0 There is a mismatch between SIZE_GENRE defined in neuralcoref/train/utils.py (SIZE_GENRE = 7) and in neuralcoref/neuralcoref.pyx (DEF SIZE_GENRE = 1). I hope it's not a misspelling... Btw if you change it to "DEF SIZE_GENRE = 7", you code works. As far as I know, your method is the only way to convert a trained neuralcoref from PyTorch to Thinc (the method suggested in #255 that uses PyTorchWrapper doesn't work with the original loading code in neuralcoref.pyx). I wish @thomwolf or someone else behind the development of this great tool will share an official way to use our beloved trained models!