Error while trying to load the model

[firasfrikha]

Hello everyone, After training the model, I have tried to load the model, when I execute this command" neuralcoref.add_to_pipe(nlp)" this error pops out :

`ExtraData Traceback (most recent call last)

in ----> 1 neuralcoref.add_to_pipe(nlp) ~/neuralcoref/neuralcoref/__init__.py in add_to_pipe(nlp, **kwargs) 40 41 def add_to_pipe(nlp, **kwargs): ---> 42 coref = NeuralCoref(nlp.vocab, **kwargs) 43 nlp.add_pipe(coref, name="neuralcoref") 44 return nlp neuralcoref.pyx in neuralcoref.neuralcoref.NeuralCoref.__init__() neuralcoref.pyx in neuralcoref.neuralcoref.NeuralCoref.from_disk() ~/anaconda3/envs/coref1/lib/python3.8/site-packages/thinc/neural/_classes/model.py in from_bytes(self, bytes_data) 353 354 def from_bytes(self, bytes_data): --> 355 data = srsly.msgpack_loads(bytes_data) 356 weights = data[b"weights"] 357 queue = [self] ~/anaconda3/envs/coref1/lib/python3.8/site-packages/srsly/_msgpack_api.py in msgpack_loads(data, use_list) 27 # msgpack-python docs suggest disabling gc before unpacking large messages 28 gc.disable() ---> 29 msg = msgpack.loads(data, raw=False, use_list=use_list) 30 gc.enable() 31 return msg ~/anaconda3/envs/coref1/lib/python3.8/site-packages/srsly/msgpack/__init__.py in unpackb(packed, **kwargs) 58 object_hook = kwargs.get('object_hook') 59 kwargs['object_hook'] = functools.partial(_decode_numpy, chain=object_hook) ---> 60 return _unpackb(packed, **kwargs) 61 62 _unpacker.pyx in srsly.msgpack._unpacker.unpackb() ExtraData: unpack(b) received extra data. ` Does anyone have a solution for this error please ?

[svlandeg]

Can you produce a minimal script that shows the error - preferably with some sample/dummy data?

[firasfrikha]

hello @svlandeg , i have resoved this problem, but now when i'm loading the model and after adding it to the pipeline, when i try to test the model , for example when i run this command

>doc = nlp('je suis Firas Frikha')

I get this error :

---------------------------------------------------------------------------
ShapeMismatchError                        Traceback (most recent call last)
<ipython-input-11-3c410e4249d4> in <module>
----> 1 doc = nlp('je suis Firas Frikdfha')

~/anaconda3/envs/coreference/lib/python3.8/site-packages/spacy/language.py in __call__(self, text, disable, component_cfg)
    447             if not hasattr(proc, "__call__"):
    448                 raise ValueError(Errors.E003.format(component=type(proc), name=name))
--> 449             doc = proc(doc, **component_cfg.get(name, {}))
    450             if doc is None:
    451                 raise ValueError(Errors.E005.format(name=name))

neuralcoref.pyx in neuralcoref.neuralcoref.NeuralCoref.__call__()

neuralcoref.pyx in neuralcoref.neuralcoref.NeuralCoref.predict()

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/neural/_classes/model.py in __call__(self, x)
    165             Must match expected shape
    166         """
--> 167         return self.predict(x)
    168 
    169     def pipe(self, stream, batch_size=128):

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/neural/_classes/feed_forward.py in predict(self, X)
     38     def predict(self, X):
     39         for layer in self._layers:
---> 40             X = layer(X)
     41         return X
     42 

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/neural/_classes/model.py in __call__(self, x)
    165             Must match expected shape
    166         """
--> 167         return self.predict(x)
    168 
    169     def pipe(self, stream, batch_size=128):

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/check.py in checked_function(wrapped, instance, args, kwargs)
    153                 if not isinstance(check, Callable):
    154                     raise ExpectedTypeError(check, ["Callable"])
--> 155                 check(arg_id, fix_args, kwargs)
    156         return wrapped(*args, **kwargs)
    157 

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/check.py in has_shape_inner(arg_id, args, kwargs)
     72             # Allow underspecified dimensions
     73             if dim is not None and arg.shape[i] != dim:
---> 74                 raise ShapeMismatchError(arg.shape, shape_values, shape)
     75 
     76     return has_shape_inner

ShapeMismatchError: 

  Shape mismatch: input (0, 668) not compatible with [None, 3924].

  Traceback:
  ├─ __call__ in /home/firas/anaconda3/envs/coreference/lib/python3.8/site-packages/spacy/language.py:449
  ├─── __call__ in neural/_classes/model.py:167
  └───── predict in neural/_classes/feed_forward.py:40
         >>> X = layer(X)

In training, ihave used fastext embedding for french:
the size embedding of the vector is 300 i changed all the constants in the utils.py but this error keep persisting.

I hope if you can help, because i'm stuck on this error for weeks now,

Thank you

[svlandeg]

As the original issue is resolved, and the new issue is a duplicate from #269, I'll close this specific issue to keep the discussion in one place.

[aditya624]

hello @svlandeg , i have resoved this problem, but now when i'm loading the model and after adding it to the pipeline, when i try to test the model , for example when i run this command

>doc = nlp('je suis Firas Frikha')

I get this error :

---------------------------------------------------------------------------
ShapeMismatchError                        Traceback (most recent call last)
<ipython-input-11-3c410e4249d4> in <module>
----> 1 doc = nlp('je suis Firas Frikdfha')

~/anaconda3/envs/coreference/lib/python3.8/site-packages/spacy/language.py in __call__(self, text, disable, component_cfg)
    447             if not hasattr(proc, "__call__"):
    448                 raise ValueError(Errors.E003.format(component=type(proc), name=name))
--> 449             doc = proc(doc, **component_cfg.get(name, {}))
    450             if doc is None:
    451                 raise ValueError(Errors.E005.format(name=name))

neuralcoref.pyx in neuralcoref.neuralcoref.NeuralCoref.__call__()

neuralcoref.pyx in neuralcoref.neuralcoref.NeuralCoref.predict()

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/neural/_classes/model.py in __call__(self, x)
    165             Must match expected shape
    166         """
--> 167         return self.predict(x)
    168 
    169     def pipe(self, stream, batch_size=128):

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/neural/_classes/feed_forward.py in predict(self, X)
     38     def predict(self, X):
     39         for layer in self._layers:
---> 40             X = layer(X)
     41         return X
     42 

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/neural/_classes/model.py in __call__(self, x)
    165             Must match expected shape
    166         """
--> 167         return self.predict(x)
    168 
    169     def pipe(self, stream, batch_size=128):

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/check.py in checked_function(wrapped, instance, args, kwargs)
    153                 if not isinstance(check, Callable):
    154                     raise ExpectedTypeError(check, ["Callable"])
--> 155                 check(arg_id, fix_args, kwargs)
    156         return wrapped(*args, **kwargs)
    157 

~/anaconda3/envs/coreference/lib/python3.8/site-packages/thinc/check.py in has_shape_inner(arg_id, args, kwargs)
     72             # Allow underspecified dimensions
     73             if dim is not None and arg.shape[i] != dim:
---> 74                 raise ShapeMismatchError(arg.shape, shape_values, shape)
     75 
     76     return has_shape_inner

ShapeMismatchError: 

  Shape mismatch: input (0, 668) not compatible with [None, 3924].

  Traceback:
  ├─ __call__ in /home/firas/anaconda3/envs/coreference/lib/python3.8/site-packages/spacy/language.py:449
  ├─── __call__ in neural/_classes/model.py:167
  └───── predict in neural/_classes/feed_forward.py:40
         >>> X = layer(X)

In training, ihave used fastext embedding for french: the size embedding of the vector is 300 i changed all the constants in the utils.py but this error keep persisting.

I hope if you can help, because i'm stuck on this error for weeks now,

Thank you

any update for this ?