search

RUCAIBox / TextBox

TextBox 2.0 is a text generation library with pre-trained language models
https://github.com/RUCAIBox/TextBox
MIT License
1.07k stars 117 forks source link

ModuleNotFoundError: No module named 'transformers.models.unilm' #318

Closed uuzna closed 1 year ago

uuzna commented 1 year ago

运行时报错ModuleNotFoundError: No module named 'transformers.models.unilm', 排查后发现是textbox\utils\utils.py文件中 from transformers.models.unilm.tokenization_unilm import UnilmTokenizer from transformers.models.mass.tokenization_mass import MassTokenizer 出错,似乎无法从transformers中导入UnilmTokenizer和MassTokenizer,该文件报错内容为: Import "transformers.models.unilm.tokenization_unilm" could not be resolved Import "transformers.models.mass.tokenization_mass" could not be resolved 我尝试升级transformers,重下transformers都未能解决

StevenTang1998 commented 1 year ago

你好，请使用install.sh进行安装，我们我们自定义了transformer的一些模型

uuzna commented 1 year ago

成功解决了,谢谢

srijith9862 commented 1 year ago

你好，请使用install.sh进行安装，我们我们自定义了transformer的一些模型

If I go with install.sh and the transformers mentioned the RUCAIBox, i'm getting the error:

File "run_textbox.py", line 2, in from textbox import run_textbox File "/content/TextBox/textbox/init.py", line 4, in from textbox.utils.enum_type import PLM_MODELS, CLM_MODELS, SEQ2SEQ_MODELS, SpecialTokens, RNN_MODELS File "/content/TextBox/textbox/utils/init.py", line 1, in from textbox.utils.logger import init_logger File "/content/TextBox/textbox/utils/logger.py", line 12, in from accelerate.logging import get_logger File "/usr/local/lib/python3.8/dist-packages/accelerate/init.py", line 7, in from .accelerator import Accelerator File "/usr/local/lib/python3.8/dist-packages/accelerate/accelerator.py", line 27, in from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state File "/usr/local/lib/python3.8/dist-packages/accelerate/checkpointing.py", line 24, in from .utils import ( File "/usr/local/lib/python3.8/dist-packages/accelerate/utils/init.py", line 103, in from .megatron_lm import ( File "/usr/local/lib/python3.8/dist-packages/accelerate/utils/megatron_lm.py", line 32, in from transformers.modeling_outputs import ( ModuleNotFoundError: No module named 'transformers.modeling_outputs'

StevenTang1998 commented 1 year ago

Could you please offer the command line when you run textbox?

srijith9862 commented 1 year ago

The command line was the same one you have mentioned in the readme.md file. python run_textbox.py --model=BART --dataset=samsum --model_path=facebook/bart-base

StevenTang1998 commented 1 year ago

could you give us the results when you run pip list in your command line.

srijith9862 commented 1 year ago

Package Version

absl-py 1.3.0 accelerate 0.15.0 aeppl 0.0.33 aesara 2.7.9 aiohttp 3.8.3 aiosignal 1.3.1 alabaster 0.7.12 albumentations 1.2.1 altair 4.2.0 appdirs 1.4.4 arviz 0.12.1 astor 0.8.1 astropy 4.3.1 astunparse 1.6.3 async-timeout 4.0.2 atari-py 0.2.9 atomicwrites 1.4.1 attrs 22.2.0 audioread 3.0.0 autograd 1.5 Babel 2.11.0 backcall 0.2.0 beautifulsoup4 4.6.3 bert-score 0.3.12 bleach 5.0.1 blis 0.7.9 bokeh 2.3.3 boto3 1.26.59 botocore 1.29.59 branca 0.6.0 bs4 0.0.1 CacheControl 0.12.11 cachetools 5.2.1 catalogue 2.0.8 certifi 2022.12.7 cffi 1.15.1 cftime 1.6.2 chardet 4.0.0 charset-normalizer 2.1.1 click 7.1.2 clikit 0.6.2 cloudpickle 2.2.0 cmake 3.22.6 cmdstanpy 1.0.8 colorama 0.4.6 colorcet 3.0.1 colorlover 0.3.0 community 1.0.0b1 confection 0.0.3 cons 0.4.5 contextlib2 0.5.5 convertdate 2.4.0 crashtest 0.3.1 crcmod 1.7 cufflinks 0.17.3 cvxopt 1.3.0 cvxpy 1.2.3 cycler 0.11.0 cymem 2.0.7 Cython 0.29.33 daft 0.0.4 dask 2022.2.1 datascience 0.17.5 db-dtypes 1.0.5 dbus-python 1.2.16 debugpy 1.0.0 decorator 4.4.2 defusedxml 0.7.1 descartes 1.1.0 dill 0.3.6 distributed 2022.2.1 dlib 19.24.0 dm-tree 0.1.8 dnspython 2.2.1 docker-pycreds 0.4.0 docutils 0.16 dopamine-rl 1.0.5 earthengine-api 0.1.335 easydict 1.10 ecos 2.0.12 editdistance 0.5.3 en-core-web-sm 3.4.1 entrypoints 0.4 ephem 4.1.4 et-xmlfile 1.1.0 etils 1.0.0 etuples 0.3.8 fa2 0.3.5 fast-bleu 0.0.90 fastai 2.7.10 fastcore 1.5.27 fastdownload 0.0.7 fastdtw 0.3.4 fastjsonschema 2.16.2 fastprogress 1.0.3 fastrlock 0.8.1 feather-format 0.4.1 filelock 3.9.0 files2rouge 2.1.0 firebase-admin 5.3.0 fix-yahoo-finance 0.0.22 Flask 1.1.4 flatbuffers 1.12 folium 0.12.1.post1 frozenlist 1.3.3 fsspec 2022.11.0 future 0.16.0 gast 0.4.0 GDAL 3.0.4 gdown 4.4.0 gensim 3.6.0 geographiclib 1.52 geopy 1.17.0 gin-config 0.5.0 gitdb 4.0.10 GitPython 3.1.30 glob2 0.7 google 2.0.3 google-api-core 2.11.0 google-api-python-client 2.70.0 google-auth 2.16.0 google-auth-httplib2 0.1.0 google-auth-oauthlib 0.4.6 google-cloud-bigquery 3.4.1 google-cloud-bigquery-storage 2.17.0 google-cloud-core 2.3.2 google-cloud-datastore 2.11.1 google-cloud-firestore 2.7.3 google-cloud-language 2.6.1 google-cloud-storage 2.7.0 google-cloud-translate 3.8.4 google-colab 1.0.0 google-crc32c 1.5.0 google-pasta 0.2.0 google-resumable-media 2.4.0 googleapis-common-protos 1.58.0 googledrivedownloader 0.4 graphviz 0.10.1 greenlet 2.0.1 grpcio 1.51.1 grpcio-status 1.48.2 gspread 3.4.2 gspread-dataframe 3.0.8 gym 0.25.2 gym-notices 0.0.8 h5py 3.1.0 HeapDict 1.0.1 hijri-converter 2.2.4 holidays 0.18 holoviews 1.14.9 html5lib 1.0.1 httpimport 0.5.18 httplib2 0.17.4 httpstan 4.6.1 huggingface-hub 0.12.0 humanize 0.5.1 hyperopt 0.2.7 idna 2.10 imageio 2.9.0 imagesize 1.4.1 imbalanced-learn 0.8.1 imblearn 0.0 imgaug 0.4.0 importlib-metadata 6.0.0 importlib-resources 5.10.2 imutils 0.5.4 inflect 2.1.0 intel-openmp 2023.0.0 intervaltree 2.1.0 ipykernel 5.3.4 ipython 7.9.0 ipython-genutils 0.2.0 ipython-sql 0.3.9 ipywidgets 7.7.1 itsdangerous 1.1.0 jax 0.3.25 jaxlib 0.3.25+cuda11.cudnn805 jieba 0.42.1 Jinja2 2.11.3 jmespath 1.0.1 joblib 1.2.0 jpeg4py 0.1.4 jsonschema 4.3.3 jupyter-client 6.1.12 jupyter-console 6.1.0 jupyter_core 5.1.3 jupyterlab-widgets 3.0.5 kaggle 1.5.12 kapre 0.3.7 keras 2.9.0 Keras-Preprocessing 1.1.2 keras-vis 0.4.1 kiwisolver 1.4.4 korean-lunar-calendar 0.3.1 langcodes 3.3.0 libclang 15.0.6.1 librosa 0.8.1 lightgbm 2.2.3 llvmlite 0.39.1 lmdb 0.99 locket 1.0.0 logical-unification 0.4.5 LunarCalendar 0.0.9 lxml 4.9.2 Markdown 3.4.1 MarkupSafe 2.0.1 marshmallow 3.19.0 matplotlib 3.2.2 matplotlib-venn 0.11.7 miniKanren 1.0.3 missingno 0.5.1 mistune 0.8.4 mizani 0.7.3 mkl 2019.0 mlxtend 0.14.0 more-itertools 9.0.0 moviepy 0.2.3.5 mpmath 1.2.1 msgpack 1.0.4 multidict 6.0.4 multipledispatch 0.6.0 multitasking 0.0.11 murmurhash 1.0.9 music21 5.5.0 natsort 5.5.0 nbconvert 5.6.1 nbformat 5.7.1 netCDF4 1.6.2 networkx 3.0 nibabel 3.0.2 nltk 3.7 notebook 5.7.16 numba 0.56.4 numexpr 2.8.4 numpy 1.21.6 oauth2client 4.1.3 oauthlib 3.2.2 okgrade 0.4.3 opencv-contrib-python 4.6.0.66 opencv-python 4.6.0.66 opencv-python-headless 4.7.0.68 openpyxl 3.0.10 opt-einsum 3.3.0 osqp 0.6.2.post0 packaging 21.3 palettable 3.3.0 pandas 1.3.5 pandas-datareader 0.9.0 pandas-gbq 0.17.9 pandas-profiling 1.4.1 pandocfilters 1.5.0 panel 0.12.1 param 1.12.3 parso 0.8.3 partd 1.3.0 pastel 0.2.1 pathlib 1.0.1 pathtools 0.1.2 pathy 0.10.1 patsy 0.5.3 pep517 0.13.0 pexpect 4.8.0 pickleshare 0.7.5 Pillow 7.1.2 pip 22.0.4 pip-tools 6.6.2 platformdirs 2.6.2 plotly 5.5.0 plotnine 0.8.0 pluggy 0.7.1 pooch 1.6.0 portalocker 2.7.0 portpicker 1.3.9 prefetch-generator 1.0.3 preshed 3.0.8 prettytable 3.6.0 progressbar2 3.38.0 prometheus-client 0.15.0 promise 2.3 prompt-toolkit 2.0.10 prophet 1.1.1 proto-plus 1.22.2 protobuf 3.19.6 psutil 5.4.8 psycopg2 2.9.5 ptyprocess 0.7.0 py 1.11.0 py4j 0.10.9.7 pyarrow 9.0.0 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycocoevalcap 1.2 pycocotools 2.0.6 pycparser 2.21 pyct 0.4.8 pydantic 1.10.4 pydata-google-auth 1.5.0 pydot 1.3.0 pydot-ng 2.0.0 pydotplus 2.0.2 PyDrive 1.3.1 pyemd 0.5.1 pyerfa 2.0.0.1 Pygments 2.6.1 PyGObject 3.36.0 pylev 1.4.0 pymc 4.1.4 PyMeeus 0.5.12 pymongo 4.3.3 pymystem3 0.2.0 PyOpenGL 3.1.6 pyparsing 3.0.9 pyrouge 0.1.3 pyrsistent 0.19.3 pysimdjson 3.2.0 PySocks 1.7.1 pystan 3.3.0 pytest 3.6.4 python-apt 2.0.1 python-dateutil 2.8.2 python-louvain 0.16 python-slugify 7.0.0 python-utils 3.4.5 pytorch-transformers 1.0.0 pytz 2022.7 pyviz-comms 2.2.1 PyWavelets 1.4.1 PyYAML 6.0 pyzmq 23.2.1 qdldl 0.1.5.post2 qudida 0.0.4 regex 2022.6.2 requests 2.25.1 requests-oauthlib 1.3.1 requests-unixsocket 0.2.0 resampy 0.4.2 rouge 1.0.1 rouge-score 0.1.2 rpy2 3.5.5 rsa 4.9 s3transfer 0.6.0 sacrebleu 2.3.1 scikit-image 0.18.3 scikit-learn 1.0.2 scipy 1.7.3 screen-resolution-extra 0.0.0 scs 3.2.2 seaborn 0.11.2 Send2Trash 1.8.0 sentencepiece 0.1.97 sentry-sdk 1.14.0 setproctitle 1.3.2 setuptools 57.4.0 setuptools-git 1.2 shapely 2.0.0 six 1.15.0 sklearn-pandas 1.8.0 smart-open 6.3.0 smmap 5.0.0 snowballstemmer 2.2.0 sortedcontainers 2.4.0 soundfile 0.11.0 spacy 3.4.4 spacy-legacy 3.0.11 spacy-loggers 1.0.4 Sphinx 3.5.4 sphinxcontrib.applehelp 1.0.3 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 SQLAlchemy 1.4.46 sqlparse 0.4.3 srsly 2.4.5 statsmodels 0.12.2 sympy 1.7.1 tables 3.7.0 tabulate 0.8.10 tblib 1.7.0 tenacity 8.1.0 tensorboard 2.9.1 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorflow 2.9.2 tensorflow-datasets 4.8.1 tensorflow-estimator 2.9.0 tensorflow-gcs-config 2.9.1 tensorflow-hub 0.12.0 tensorflow-io-gcs-filesystem 0.29.0 tensorflow-metadata 1.12.0 tensorflow-probability 0.17.0 termcolor 2.2.0 terminado 0.13.3 testpath 0.6.0 text-unidecode 1.3 textblob 0.15.3 thinc 8.1.6 threadpoolctl 3.1.0 tifffile 2022.10.10 tokenizers 0.13.2 toml 0.10.2 tomli 2.0.1 toolz 0.12.0 torch 1.13.1+cu116 torchaudio 0.13.1+cu116 torchsummary 1.5.1 torchtext 0.14.1 torchvision 0.14.1+cu116 tornado 6.0.4 tqdm 4.64.1 traitlets 5.7.1 tweepy 3.10.0 typeguard 2.7.1 typer 0.7.0 typing 3.7.4.3 typing_extensions 4.4.0 tzlocal 1.5.1 uritemplate 4.1.1 urllib3 1.26.14 vega-datasets 0.9.0 wandb 0.13.9 wasabi 0.10.1 wcwidth 0.2.5 webargs 8.2.0 webencodings 0.5.1 Werkzeug 1.0.1 wheel 0.38.4 widgetsnbextension 3.6.1 wordcloud 1.8.2.2 wrapt 1.14.1 xarray 2022.12.0 xarray-einstats 0.4.0 xgboost 0.90 xkit 0.0.0 xlrd 1.2.0 xlwt 1.3.0 yarl 1.8.2 yellowbrick 1.5 zict 2.2.0 zipp 3.11.0

StevenTang1998 commented 1 year ago

It seems that you didn't install transformers well.

You may need to install it again:

git clone https://github.com/RUCAIBox/transformers.git
cd transformers
pip install -e .
srijith9862 commented 1 year ago

git clone https://github.com/RUCAIBox/transformers.git cd transformers pip install -e .

still getting the ame error as before: File "run_textbox.py", line 2, in from textbox import run_textbox File "/content/TextBox/textbox/init.py", line 4, in from textbox.utils.enum_type import PLM_MODELS, CLM_MODELS, SEQ2SEQ_MODELS, SpecialTokens, RNN_MODELS File "/content/TextBox/textbox/utils/init.py", line 1, in from textbox.utils.logger import init_logger File "/content/TextBox/textbox/utils/logger.py", line 12, in from accelerate.logging import get_logger File "/usr/local/lib/python3.8/dist-packages/accelerate/init.py", line 7, in from .accelerator import Accelerator File "/usr/local/lib/python3.8/dist-packages/accelerate/accelerator.py", line 27, in from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state File "/usr/local/lib/python3.8/dist-packages/accelerate/checkpointing.py", line 24, in from .utils import ( File "/usr/local/lib/python3.8/dist-packages/accelerate/utils/init.py", line 103, in from .megatron_lm import ( File "/usr/local/lib/python3.8/dist-packages/accelerate/utils/megatron_lm.py", line 32, in from transformers.modeling_outputs import ( ModuleNotFoundError: No module named 'transformers.modeling_outputs'

There is no transformers module getting created only pytorch-transformers - 1.0.0 is only there

StevenTang1998 commented 1 year ago

Maybe you can open a Python window and see whether import transformers can work well.

srijith9862 commented 1 year ago

Still, it isn't working but if I download the latest transformers through pip install transformers. Then run the command: python run_textbox.py --model=BART --dataset=samsum --model_path=facebook/bart-base then I'm getting the error: File "run_textbox.py", line 2, in from textbox import run_textbox File "/content/TextBox/textbox/init.py", line 4, in from textbox.utils.enum_type import PLM_MODELS, CLM_MODELS, SEQ2SEQ_MODELS, SpecialTokens, RNN_MODELS File "/content/TextBox/textbox/utils/init.py", line 1, in from textbox.utils.logger import init_logger File "/content/TextBox/textbox/utils/logger.py", line 13, in from textbox.utils.utils import ensure_dir File "/content/TextBox/textbox/utils/utils.py", line 13, in from transformers.models.unilm.tokenization_unilm import UnilmTokenizer ModuleNotFoundError: No module named 'transformers.models.unilm'

StevenTang1998 commented 1 year ago

If so, you could provide the whole install information when you run these commands:

git clone https://github.com/RUCAIBox/transformers.git
cd transformers
pip install -e .
srijith9862 commented 1 year ago

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Obtaining file:///content/TextBox/transformers Installing build dependencies ... done Checking if build backend supports build_editable ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (4.64.1) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (0.13.2) Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (3.9.0) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (1.21.6) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (6.0) Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (2.25.1) Requirement already satisfied: huggingface-hub<1.0,>=0.10.0 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (0.12.0) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (2022.6.2) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers==4.26.0.dev0) (21.3) Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0,>=0.10.0->transformers==4.26.0.dev0) (4.4.0) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->transformers==4.26.0.dev0) (3.0.9) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers==4.26.0.dev0) (2022.12.7) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers==4.26.0.dev0) (2.10) Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers==4.26.0.dev0) (4.0.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers==4.26.0.dev0) (1.26.14) Installing collected packages: transformers Running setup.py develop for transformers Successfully installed transformers

StevenTang1998 commented 1 year ago

Maybe you can try to replace pip install -e . with pip install ..

srijith9862 commented 1 year ago

The problem of ModuleNotFoundError: No module named 'transformers.models.unilm' still persists.

If possible, Can you provide me with a colab or jupiter notebook where the model is working? Till the command: python run_textbox.py --model=BART --dataset=samsum --model_path=facebook/bart-base

StevenTang1998 commented 1 year ago

Sorry, we do not test our codes using colab or jupiter. We guess the fail installation may be due to the colab environment.

I have one last solution you can try:

pip uninstall transformers
pip install git+https://github.com/RUCAIBox/transformers.git
srijith9862 commented 1 year ago

It still not working but let me try it out.

Would love to know where you would test your codes.

StevenTang1998 commented 1 year ago

We run our code on the local machine with Ubuntu 20.04.

srijith9862 commented 1 year ago

It's the same on my local machine. I was getting the ModuleNotFoundError: No module named 'transformers.models.unilm' .

srijith9862 commented 1 year ago

But let me try it in my local machine with:

pip uninstall transformers pip install git+https://github.com/RUCAIBox/transformers.git

StevenTang1998 commented 1 year ago

When I run pip list, it will show the transformers is installed: 

transformers            4.26.0.dev0 /home/TextBox/transformers