AttributeError: 'FeedForward' object has no attribute 'G'

[damianoporta]

Hi, i am training a new NER model ad i found a problem with conv_depth. I recently added another similar issue, this: https://github.com/explosion/spaCy/issues/4058 that has been closed,

Basically i only have added this:

    ner = nlp.create_pipe("ner", config={"conv_depth": 6})
    nlp.add_pipe(ner, last=True)

as @ines said. When i try to load the model again i get the following error:

>>> nlp = spacy.load('/home/nlp/32')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy/__init__.py", line 27, in load
    return util.load_model(name, **overrides)
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy/util.py", line 133, in load_model
    return load_model_from_path(Path(name), **overrides)
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy/util.py", line 173, in load_model_from_path
    return nlp.from_disk(model_path)
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy/language.py", line 791, in from_disk
    util.from_disk(path, deserializers, exclude)
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy/util.py", line 630, in from_disk
    reader(path / key)
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy/language.py", line 787, in <lambda>
    deserializers[name] = lambda p, proc=proc: proc.from_disk(p, exclude=["vocab"])
  File "nn_parser.pyx", line 634, in spacy.syntax.nn_parser.Parser.from_disk
  File "/home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/thinc/neural/_classes/model.py", line 371, in from_bytes
    dest = getattr(layer, name)
AttributeError: 'FeedForward' object has no attribute 'G'

in this case the trick with _os.environ['convdepth'] = '6' does not solve the problem. I also have checked the ner/cfg and this is the content:

{
  "conv_depth":6,
  "beam_width":1,
  "beam_density":0.0,
  "beam_update_prob":1.0,
  "cnn_maxout_pieces":3,
  "nr_class":81,
  "hidden_depth":1,
  "token_vector_width":96,
  "hidden_width":64,
  "maxout_pieces":2,
  "pretrained_vectors":"it_model.vectors",
  "bilstm_depth":0
}

The model has been trained with the same code:

other_pipes = [pipe for pipe in nlp.pipe_names if pipe != "ner"]

with nlp.disable_pipes(*other_pipes):  # only train NER
    # reset and initialize the weights randomly – but only if we're
    # training a new model
    nlp.begin_training()

    for itn in range(N_ITER):
        random.shuffle(TRAIN_DATA)
        losses = {}
        # batch up the examples using spaCy's minibatch
        batches = minibatch(TRAIN_DATA, size=compounding(4.0, 32.0, 1.001))

        for batch in batches:
            texts, annotations = zip(*batch)
            nlp.update(
                texts,  # batch of texts
                annotations,  # batch of annotations
                drop=0.2,  # dropout - make it harder to memorise data
                losses=losses,
            )

I did not pass the config in _begintraining() with component_cfg={"ner": {"conv_depth": 6}} I thought it was useless because of the ner initialization:

ner = nlp.create_pipe("ner", config={"conv_depth": 6})

Do i also need it? Any workaround? Thanks

Your Environment

spaCy version    2.1.4                         
Location         /home/nlp/.prodigy-1.8.3/lib/python3.6/site-packages/spacy
Platform         Linux-4.15.0-55-generic-x86_64-with-Ubuntu-18.04-bionic
Python version   3.6.7                         
Models   

[damianoporta]

Should I set the conv_depth differently?

[FallakAsad]

I am facing the same issue.

[svlandeg]

Hi @damianoporta, apologies for the late follow-up. We recently figured out what was happening, cf https://github.com/explosion/spaCy/issues/4934#issuecomment-593389846. In short, it looks like the conv_depth setting got ignored and did not actually change the model, but the value was still stored in the output config file, which then caused an incompatibility upon IO, and a crash. PR #5078 should at least prevent the crash.

We are currently working on spaCy 3.0, where we will have a much more clean way to define these kind of hyperparameters of the ML models.

Also have a look at this advice by Matt:

By the way, adding CNN layers is unlikely to be the most effective option. You'll be better off increasing token_vector_width, and possibly installing PyTorch and increasing bilstm_depth.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.