search

explosion / spaCy

💫 Industrial-strength Natural Language Processing (NLP) in Python
https://spacy.io
MIT License
30.09k stars 4.4k forks source link

ValueError: Out shape is mismatched #11137

Closed cliuxinxin closed 2 years ago

cliuxinxin commented 2 years ago

How to reproduce the behaviour

Your Environment

Some weights of the model checkpoint at hfl/chinese-roberta-wwm-ext were not used when initializing BertModel: ['cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.seq_relationship.bias']

when I update the spacy to latest,

cant work any more

when I change back to 3.3.1 , it's working.

I packed all the code and data. Just run the cmd

task = "news" os.environ['TASK'] = task !python -m spacy project run all test.zip

danieldk commented 2 years ago

Thanks for reporting this issue. Could you include all the files necessary to reproduce the issue? For example, the conversion scripts are missing in the provided zip files.

cliuxinxin commented 2 years ago

Thank you. I packed the data in to the zip file and upload.

danieldk commented 2 years ago

Thanks! I can now run the project, but it does the initialization without any issues and proceeds to finetune the transformer model. Any chance you can post the output of pip list, so that I can try to reproduce it with the same versions of the spaCy/Thinc dependencies?

cliuxinxin commented 2 years ago

!pip install -U pip setuptools wheel !pip install -U 'spacy[transformers,lookups]' !python -m spacy download zh_core_web_lg !pip install doccano-client !pip install datasets !pip install seqeval !pip install spacy-transformers !pip install umap-learn

I use the colab

Package Version

absl-py 1.1.0 aiohttp 3.8.1 aiosignal 1.2.0 alabaster 0.7.12 albumentations 0.1.12 altair 4.2.0 appdirs 1.4.4 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 arviz 0.12.1 astor 0.8.1 astropy 4.3.1 astunparse 1.6.3 async-timeout 4.0.2 asynctest 0.13.0 atari-py 0.2.9 atomicwrites 1.4.1 attrs 21.4.0 audioread 2.1.9 autograd 1.4 Babel 2.10.3 backcall 0.2.0 beautifulsoup4 4.6.3 bleach 5.0.1 blis 0.7.8 bokeh 2.3.3 branca 0.5.0 bs4 0.0.1 CacheControl 0.12.11 cached-property 1.5.2 cachetools 4.2.4 catalogue 2.0.7 certifi 2022.6.15 cffi 1.15.1 cftime 1.6.1 chardet 3.0.4 charset-normalizer 2.1.0 click 7.1.2 cloudpickle 1.3.0 cmake 3.22.5 cmdstanpy 0.9.5 colorcet 3.0.0 colorlover 0.3.0 community 1.0.0b1 contextlib2 0.5.5 convertdate 2.4.0 coverage 3.7.1 coveralls 0.5 crcmod 1.7 cufflinks 0.17.3 cupy-cuda111 9.4.0 cvxopt 1.2.7 cvxpy 1.0.31 cycler 0.11.0 cymem 2.0.6 Cython 0.29.30 daft 0.0.4 dask 2.12.0 datascience 0.10.6 datasets 2.3.2 debugpy 1.0.0 decorator 4.4.2 defusedxml 0.7.1 deprecat 2.1.1 descartes 1.1.0 dill 0.3.5.1 distributed 1.25.3 dlib 19.18.0+zzzcolab20220513001918 dm-tree 0.1.7 doccano-client 1.0.3 docopt 0.6.2 docutils 0.17.1 dopamine-rl 1.0.5 earthengine-api 0.1.316 easydict 1.9 ecos 2.0.10 editdistance 0.5.3 en-core-web-sm 3.3.0 entrypoints 0.4 ephem 4.1.3 et-xmlfile 1.1.0 etils 0.6.0 fa2 0.3.5 fastai 2.7.6 fastcore 1.4.5 fastdownload 0.0.7 fastdtw 0.3.4 fastjsonschema 2.15.3 fastprogress 1.0.2 fastrlock 0.8 fbprophet 0.7.1 feather-format 0.4.1 filelock 3.7.1 firebase-admin 4.4.0 fix-yahoo-finance 0.0.22 Flask 1.1.4 flatbuffers 2.0 folium 0.8.3 frozenlist 1.3.0 fsspec 2022.5.0 future 0.16.0 gast 0.5.3 GDAL 2.2.2 gdown 4.4.0 gensim 3.6.0 geographiclib 1.52 geopy 1.17.0 gin-config 0.5.0 glob2 0.7 google 2.0.3 google-api-core 1.31.6 google-api-python-client 1.12.11 google-auth 1.35.0 google-auth-httplib2 0.0.4 google-auth-oauthlib 0.4.6 google-cloud-bigquery 1.21.0 google-cloud-bigquery-storage 1.1.2 google-cloud-core 1.0.3 google-cloud-datastore 1.8.0 google-cloud-firestore 1.7.0 google-cloud-language 1.2.0 google-cloud-storage 1.18.1 google-cloud-translate 1.5.0 google-colab 1.0.0 google-pasta 0.2.0 google-resumable-media 0.4.1 googleapis-common-protos 1.56.3 googledrivedownloader 0.4 graphviz 0.10.1 greenlet 1.1.2 grpcio 1.47.0 gspread 3.4.2 gspread-dataframe 3.0.8 gym 0.17.3 h5py 3.1.0 HeapDict 1.0.1 hijri-converter 2.2.4 holidays 0.10.5.2 holoviews 1.14.9 html5lib 1.0.1 httpimport 0.5.18 httplib2 0.17.4 httplib2shim 0.0.3 huggingface-hub 0.8.1 humanize 0.5.1 hyperopt 0.1.2 ideep4py 2.0.0.post3 idna 2.10 imageio 2.4.1 imagesize 1.4.1 imbalanced-learn 0.8.1 imblearn 0.0 imgaug 0.2.9 importlib-metadata 4.12.0 importlib-resources 5.8.0 imutils 0.5.4 inflect 2.1.0 iniconfig 1.1.1 intel-openmp 2022.1.0 intervaltree 2.1.0 ipykernel 4.10.1 ipython 5.5.0 ipython-genutils 0.2.0 ipython-sql 0.3.9 ipywidgets 7.7.1 itsdangerous 1.1.0 jax 0.3.14 jaxlib 0.3.14+cuda11.cudnn805 jedi 0.18.1 jieba 0.42.1 Jinja2 2.11.3 joblib 1.1.0 jpeg4py 0.1.4 jsonschema 4.3.3 jupyter 1.0.0 jupyter-client 5.3.5 jupyter-console 5.2.0 jupyter-core 4.11.1 jupyterlab-pygments 0.2.2 jupyterlab-widgets 1.1.1 kaggle 1.5.12 kapre 0.3.7 keras 2.8.0 Keras-Preprocessing 1.1.2 keras-vis 0.4.1 kiwisolver 1.4.3 korean-lunar-calendar 0.2.1 langcodes 3.3.0 libclang 14.0.1 librosa 0.8.1 lightgbm 2.2.3 llvmlite 0.34.0 lmdb 0.99 LunarCalendar 0.0.9 lxml 4.2.6 Markdown 3.3.7 MarkupSafe 2.0.1 matplotlib 3.2.2 matplotlib-inline 0.1.3 matplotlib-venn 0.11.7 missingno 0.5.1 mistune 0.8.4 mizani 0.6.0 mkl 2019.0 mlxtend 0.14.0 more-itertools 8.13.0 moviepy 0.2.3.5 mpmath 1.2.1 msgpack 1.0.4 multidict 6.0.2 multiprocess 0.70.13 multitasking 0.0.11 murmurhash 1.0.7 music21 5.5.0 natsort 5.5.0 nbclient 0.6.6 nbconvert 5.6.1 nbformat 5.4.0 nest-asyncio 1.5.5 netCDF4 1.6.0 networkx 2.6.3 nibabel 3.0.2 nltk 3.7 notebook 5.3.1 numba 0.51.2 numexpr 2.8.3 numpy 1.21.6 oauth2client 4.1.3 oauthlib 3.2.0 okgrade 0.4.3 opencv-contrib-python 4.1.2.30 opencv-python 4.1.2.30 openpyxl 3.0.10 opt-einsum 3.3.0 osqp 0.6.2.post0 packaging 21.3 palettable 3.3.0 pandas 1.3.5 pandas-datareader 0.9.0 pandas-gbq 0.13.3 pandas-profiling 1.4.1 pandocfilters 1.5.0 panel 0.12.1 param 1.12.2 parso 0.8.3 pathlib 1.0.1 pathy 0.6.2 patsy 0.5.2 pep517 0.12.0 pexpect 4.8.0 pickleshare 0.7.5 Pillow 7.1.2 pip 22.1.2 pip-tools 6.2.0 plotly 5.5.0 plotnine 0.6.0 pluggy 0.7.1 pooch 1.6.0 portpicker 1.3.9 prefetch-generator 1.0.1 preshed 3.0.6 prettytable 3.3.0 progressbar2 3.38.0 prometheus-client 0.14.1 promise 2.3 prompt-toolkit 1.0.18 protobuf 3.17.3 psutil 5.4.8 psycopg2 2.7.6.1 ptyprocess 0.7.0 py 1.11.0 pyarrow 6.0.1 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycocotools 2.0.4 pycparser 2.21 pyct 0.4.8 pydantic 1.8.2 pydata-google-auth 1.4.0 pydot 1.3.0 pydot-ng 2.0.0 pydotplus 2.0.2 PyDrive 1.3.1 pyemd 0.5.1 pyerfa 2.0.0.1 pyglet 1.5.0 Pygments 2.6.1 PyGObject 3.26.1 pymc3 3.11.5 PyMeeus 0.5.11 pymongo 4.1.1 pymystem3 0.2.0 pynndescent 0.5.7 PyOpenGL 3.1.6 pyparsing 3.0.9 pyrsistent 0.18.1 pysndfile 1.3.8 PySocks 1.7.1 pystan 2.19.1.1 pytest 3.6.4 python-apt 0.0.0 python-chess 0.23.11 python-dateutil 2.8.2 python-louvain 0.16 python-slugify 6.1.2 python-utils 3.3.3 pytz 2022.1 pyviz-comms 2.2.0 PyWavelets 1.3.0 PyYAML 6.0 pyzmq 23.2.0 qdldl 0.1.5.post2 qtconsole 5.3.1 QtPy 2.1.0 regex 2022.6.2 requests 2.23.0 requests-oauthlib 1.3.1 resampy 0.3.1 responses 0.18.0 rpy2 3.4.5 rsa 4.8 scikit-image 0.18.3 scikit-learn 1.0.2 scipy 1.7.3 screen-resolution-extra 0.0.0 scs 3.2.0 seaborn 0.11.2 semver 2.13.0 Send2Trash 1.8.0 seqeval 1.2.2 setuptools 63.2.0 setuptools-git 1.2 Shapely 1.8.2 simplegeneric 0.8.1 six 1.15.0 sklearn 0.0 sklearn-pandas 1.8.0 smart-open 5.2.1 snowballstemmer 2.2.0 sortedcontainers 2.4.0 SoundFile 0.10.3.post1 soupsieve 2.3.2.post1 spacy 3.4.0 spacy-alignments 0.8.5 spacy-legacy 3.0.9 spacy-loggers 1.0.2 spacy-lookups-data 1.0.3 spacy-pkuseg 0.0.31 spacy-transformers 1.1.7 Sphinx 1.8.6 sphinxcontrib-serializinghtml 1.1.5 sphinxcontrib-websupport 1.2.4 SQLAlchemy 1.4.39 sqlparse 0.4.2 srsly 2.4.3 statsmodels 0.10.2 sympy 1.7.1 tables 3.7.0 tabulate 0.8.10 tblib 1.7.0 tenacity 8.0.1 tensorboard 2.8.0 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorflow 2.8.2+zzzcolab20220527125636 tensorflow-datasets 4.0.1 tensorflow-estimator 2.8.0 tensorflow-gcs-config 2.8.0 tensorflow-hub 0.12.0 tensorflow-io-gcs-filesystem 0.26.0 tensorflow-metadata 1.9.0 tensorflow-probability 0.16.0 termcolor 1.1.0 terminado 0.13.3 testpath 0.6.0 text-unidecode 1.3 textblob 0.15.3 Theano-PyMC 1.1.2 thinc 8.1.0 threadpoolctl 3.1.0 tifffile 2021.11.2 tinycss2 1.1.1 tokenizers 0.12.1 tomli 2.0.1 toolz 0.12.0 torch 1.12.0+cu113 torchaudio 0.12.0+cu113 torchsummary 1.5.1 torchtext 0.13.0 torchvision 0.13.0+cu113 tornado 5.1.1 tqdm 4.64.0 traitlets 5.1.1 transformers 4.20.1 tweepy 3.10.0 typeguard 2.7.1 typer 0.4.2 typing_extensions 4.1.1 tzlocal 1.5.1 umap-learn 0.5.3 uritemplate 3.0.1 urllib3 1.25.11 vega-datasets 0.9.0 wasabi 0.9.1 wcwidth 0.2.5 webencodings 0.5.1 Werkzeug 1.0.1 wheel 0.37.1 widgetsnbextension 3.6.1 wordcloud 1.5.0 wrapt 1.14.1 xarray 0.20.2 xarray-einstats 0.2.2 xgboost 0.90 xkit 0.0.0 xlrd 1.1.0 xlwt 1.3.0 xxhash 3.0.0 yarl 1.7.2 yellowbrick 1.4 zh-core-web-lg 3.4.0 zict 2.2.0 zipp 3.8.0

danieldk commented 2 years ago

Ah, I can reproduce it when downgrading to the cupy version that you have (9.4.0). Can you try to upgrade the CuPy package?

pip install --upgrade cupy-cuda111

I still have to look what the issue is with the older CuPy version, but that might at least solve the problem for you.

cliuxinxin commented 2 years ago

image Yes, it is working. Thank you very much. Hope you can fix the older version problem

miguelwon commented 2 years ago

Hi, I'm having the same issue, although already installed cupy-cuda111. This is my log:

(env) mwon@sebruno2:~/data-mwon/TC/src$ python -m spacy train config.cfg --output ../results_train_fase_1_809sample/ --paths.train  ../data/train_val_test/conll/train_fase_1.spacy --paths.dev ../data/train_val_test/conll/dev_fase_1.spacy -g 0
ℹ Saving to output directory: ../results_train_fase_1_809sample
ℹ Using GPU: 0

=========================== Initializing pipeline ===========================
[2022-07-25 12:15:05,619] [INFO] Set up nlp object from config
[2022-07-25 12:15:05,629] [INFO] Pipeline: ['transformer', 'ner']
[2022-07-25 12:15:05,632] [INFO] Created vocabulary
[2022-07-25 12:15:05,632] [INFO] Finished initializing nlp object
Some weights of the model checkpoint at neuralmind/bert-base-portuguese-cased were not used when initializing BertModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.weight', 'cls.predictions.decoder.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/cli/train.py", line 45, in train_cli
    train(config_path, output_path, use_gpu=use_gpu, overrides=overrides)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/cli/train.py", line 72, in train
    nlp = init_nlp(config, use_gpu=use_gpu)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/training/initialize.py", line 84, in init_nlp
    nlp.initialize(lambda: train_corpus(nlp), sgd=optimizer)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/language.py", line 1317, in initialize
    proc.initialize(get_examples, nlp=self, **p_settings)
  File "spacy/pipeline/transition_parser.pyx", line 575, in spacy.pipeline.transition_parser.Parser.initialize
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/thinc/model.py", line 299, in initialize
    self.init(self, X=X, Y=Y)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/ml/tb_framework.py", line 47, in init
    lower.initialize()
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/thinc/model.py", line 299, in initialize
    self.init(self, X=X, Y=Y)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/ml/_precomputable_affine.py", line 150, in init
    acts1 = predict(ids, tokvecs)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/ml/_precomputable_affine.py", line 131, in predict
    hiddens = model.predict(tokvecs[:-1])  # (nW, f, o, p)
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/thinc/model.py", line 315, in predict
    return self._func(self, X, is_train=False)[0]
  File "/mnt/sdb/data-mwon/TC/env/lib/python3.6/site-packages/spacy/ml/_precomputable_affine.py", line 29, in forward
    Yf[0] = model.get_param("pad")
  File "cupy/_core/core.pyx", line 1418, in cupy._core.core.ndarray.__setitem__
  File "cupy/_core/_routines_indexing.pyx", line 54, in cupy._core._routines_indexing._ndarray_setitem
  File "cupy/_core/_routines_indexing.pyx", line 959, in cupy._core._routines_indexing._scatter_op
  File "cupy/_core/_kernel.pyx", line 1161, in cupy._core._kernel.ufunc.__call__
  File "cupy/_core/_kernel.pyx", line 594, in cupy._core._kernel._get_out_args
ValueError: Out shape is mismatched
danieldk commented 2 years ago

Could you try this PR: https://github.com/explosion/spaCy/pull/11194 ?

miguelwon commented 2 years ago

Thanks. It's working now.

github-actions[bot] commented 2 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.