ValueError: [E084] Error assigning label ID 4317129024397789502 to span: not in StringStore.

[baiziyuandyufei]

How to reproduce the behaviour

1. generate the base_config.cfg file, reference https://nightly.spacy.io/usage/training#quickstart

[paths]
train = null
dev = null

[system]
gpu_allocator = "pytorch"

[nlp]
lang = "zh"
pipeline = ["transformer","ner"]
tokenizer = {"@tokenizers": "spacy.Tokenizer.v1"}
batch_size = 128

[components]

[components.transformer]
factory = "transformer"

[components.transformer.model]
@architectures = "spacy-transformers.TransformerModel.v1"
name = "bert-base-chinese"
tokenizer_config = {"use_fast": true}

[components.transformer.model.get_spans]
@span_getters = "spacy-transformers.strided_spans.v1"
window = 128
stride = 96

[components.ner]
factory = "ner"

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = false
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0

[components.ner.model.tok2vec.pooling]
@layers = "reduce_mean.v1"

[corpora]

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 500

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0

[training]
accumulate_gradient = 3
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"

[training.optimizer]
@optimizers = "Adam.v1"

[training.optimizer.learn_rate]
@schedules = "warmup_linear.v1"
warmup_steps = 250
total_steps = 20000
initial_rate = 5e-5

[training.batcher]
@batchers = "spacy.batch_by_padded.v1"
discard_oversize = true
size = 2000
buffer = 256

[initialize]
vectors = null

2. fill the base_config.cft to config.cfg

python -m spacy init fill-config base_config.cfg config.cfg

3. prepare the ner-token-perline.iob corpus

海 O
钓 O
比 O
赛 O
地 O
点 O
在 O
厦 B-LOC
门 I-LOC
与 O
金 B-LOC
门 I-LOC
之 O
间 O
的 O
海 O
域 O
。 O

这 O
座 O
依 O
山 O
傍 O
水 O
的 O

4. convert the corpus to spacy format

python -m spacy convert input_file_path output_file_path -c iob -n 10

5. train the model

python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./dev.spacy

6. test the model

the test_code:

import spacy
from spacy.lang.zh import Chinese

nlp = spacy.load("output/model-best")
nlp.tokenizer = Chinese().tokenizer
doc = nlp("红楼中的贾宝玉")

the error is

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/git_code/2021-?ʷ?????/spacy/spaCy-3.0.0rc5/spacy/language.py", line 985, in __call__
    doc = proc(doc, **component_cfg.get(name, {}))
  File "spacy/pipeline/transition_parser.pyx", line 168, in spacy.pipeline.transition_parser.Parser.__call__
  File "spacy/pipeline/transition_parser.pyx", line 278, in spacy.pipeline.transition_parser.Parser.set_annotations
  File "spacy/pipeline/_parser_internals/ner.pyx", line 243, in spacy.pipeline._parser_internals.ner.BiluoPushDown.set_annotations
  File "spacy/tokens/span.pyx", line 105, in spacy.tokens.span.Span.__cinit__
ValueError: [E084] Error assigning label ID 4317129024397789502 to span: not in StringStore.

Your Environment

• Operating System: Linux-4.19.121-linuxkit-x86_64-with-Ubuntu-18.04-bionic
• Python Version Used: 3.6.9
• spaCy Version Used: 3.0.0rc5
• Environment Information: Package Version attrs 20.3.0 blis 0.7.4 catalogue 2.0.1 certifi 2020.12.5 chardet 4.0.0 click 7.1.2 contextvars 2.4 cymem 2.0.5 Cython 0.29.21 dataclasses 0.8 filelock 3.0.12 flake8 3.5.0 ftfy 5.8 hypothesis 6.0.4 idna 2.10 immutables 0.14 importlib-metadata 3.4.0 iniconfig 1.1.1 Jinja2 2.11.2 joblib 1.0.0 MarkupSafe 1.1.1 mccabe 0.6.1 ml-datasets 0.2.0a0 mock 2.0.0 murmurhash 1.0.5 numpy 1.19.5 packaging 20.8 pathy 0.3.4 pbr 5.5.1 pip 21.0 pkg-resources 0.0.0 pluggy 0.13.1 preshed 3.0.5 py 1.10.0 pycodestyle 2.3.1 pydantic 1.7.3 pyflakes 1.6.0 pyparsing 2.4.7 pytest 6.2.2 pytest-timeout 1.4.2 regex 2020.11.13 requests 2.25.1 sacremoses 0.0.43 setuptools 52.0.0 six 1.15.0 smart-open 3.0.0 sortedcontainers 2.3.0 spacy-alignments 0.7.2 spacy-legacy 3.0.0 spacy-nightly 3.0.0rc5 spacy-pkuseg 0.0.28 spacy-transformers 1.0.0rc3 srsly 2.3.2 thinc 8.0.1 tokenizers 0.9.4 toml 0.10.2 torch 1.7.1 torchcontrib 0.0.2 tqdm 4.56.0 transformers 4.2.2 typer 0.3.2 typing-extensions 3.7.4.3 urllib3 1.26.3 wasabi 0.8.1 wcwidth 0.2.5 wheel 0.36.2 zh-core-web-trf 3.0.0a1 zipp 3.4.0

[adrianeboyd]

Try leaving out this line? The default tokenizer should be the character-based one and replacing it could be leading to the problems with the vocab:

nlp.tokenizer = Chinese().tokenizer

[baiziyuandyufei]

@adrianeboyd if don't set nlp.tokenizer， then the doc = nlp("红楼中的贾宝玉") print([token for token in doc]) will output ['红楼中的贾宝玉']

[adrianeboyd]

Hmm, the tokenizer should be configured correctly in the config file or mismatches between training vs. runtime tokenization will cause problems with the models anyway. What does the evaluation look like while training?

[adrianeboyd]

Ah, I think the tokenizer config is incorrect, so I doubt the model trained correctly, either. The config should look like this:

[nlp]
lang = "zh"

...

[nlp.tokenizer]
@tokenizers = "spacy.zh.ChineseTokenizer"
segmenter = "char"

...

[initialize.tokenizer]
pkuseg_model = null
pkuseg_user_dict = "default"

We need to fix the quickstart here, it looks like it doesn't handle the language-specific tokenizers correctly.

If you want to see the basics for the tokenizer config, you can also try this:

from spacy.lang.zh import Chinese
nlp = Chinese()
print(nlp.config.to_str())

[adrianeboyd]

Okay, #6870 should fix this in the quickstart template. The changes should be on the website later today, thanks for the bug report!

[baiziyuandyufei]

@adrianeboyd Thank you! after modify the base_config.cfg generated by https://spacy.io/usage/training, then it work well. The modified base_config.cfg below

[paths]
train = null
dev = null

[system]
gpu_allocator = "pytorch"

[nlp]
lang = "zh"
pipeline = ["transformer","ner"]
# tokenizer = {"@tokenizers": "spacy.Tokenizer.v1"}
batch_size = 128

[nlp.tokenizer]
@tokenizers = "spacy.zh.ChineseTokenizer"
segmenter = char

[components]

[components.transformer]
factory = "transformer"

[components.transformer.model]
@architectures = "spacy-transformers.TransformerModel.v1"
name = "bert-base-chinese"
tokenizer_config = {"use_fast": true}

[components.transformer.model.get_spans]
@span_getters = "spacy-transformers.strided_spans.v1"
window = 128
stride = 96

[components.ner]
factory = "ner"

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = false
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0

[components.ner.model.tok2vec.pooling]
@layers = "reduce_mean.v1"

[corpora]

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 500

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0

[training]
accumulate_gradient = 3
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"

[training.optimizer]
@optimizers = "Adam.v1"

[training.optimizer.learn_rate]
@schedules = "warmup_linear.v1"
warmup_steps = 250
total_steps = 20000
initial_rate = 5e-5

[training.batcher]
@batchers = "spacy.batch_by_padded.v1"
discard_oversize = true
size = 2000
buffer = 256

[initialize]
vectors = null

[initialize.tokenizer]
pkuseg_model = null
pkuseg_user_dict = "default"

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.