Getting truncated "generated text"

[nazkhan-8451]

[nazkhan-8451]

Without changing anything sometimes The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:0 for open-end generation. this message comes on I get the following truncated output: Out[2]: [{'generated_text': 'Explain to me the difference between nuclear fission and fusion. \n nuclear fission: splitting of an'}]

When there is no `The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:0 for open-end generation. message, I get the full response. What is happening? Running on Azure Databricks.

[FurkanGozukara]

What are your system library properties

Such as torch version python version etc

[nazkhan-8451]

absl-py==1.0.0 accelerate==0.18.0 aiohttp==3.8.4 aiosignal==1.3.1 antlr4-python3-runtime==4.9.3 anyio==3.6.2 argilla==1.6.0 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 astor==0.8.1 asttokens==2.0.5 astunparse==1.6.3 async-timeout==4.0.2 attrs==21.4.0 autocfg==0.0.8 autogluon==0.5.0 autogluon-contrib-nlp==0.0.1b20220208 autogluon.common==0.5.0 autogluon.core==0.5.0 autogluon.features==0.5.0 autogluon.multimodal==0.5.0 autogluon.tabular==0.5.0 autogluon.text==0.5.0 autogluon.timeseries==0.5.0 autogluon.vision==0.5.0 azure-core==1.26.3 azure-cosmos==4.2.0 backcall==0.2.0 backoff==2.2.1 backports.entry-points-selectable==1.2.0 bcrypt==3.2.0 beautifulsoup4==4.11.1 black==22.3.0 bleach==4.1.0 blis==0.7.9 boto3==1.21.32 botocore==1.24.32 cachetools==4.2.2 catalogue==2.0.8 catboost==1.0.6 category-encoders==2.5.1.post0 ccds==2.0.2 cdadata==0.31.124104 certifi==2022.12.7 cffi==1.15.0 chardet==4.0.0 charset-normalizer==2.0.4 click==8.0.4 cloudpickle==2.0.0 cmdstanpy==1.1.0 colorama==0.4.6 coloredlogs==15.0.1 commonmark==0.9.1 component-logger==1.11.1.post0 confection==0.0.4 configparser==5.2.0 contextvars==2.4 convertdate==2.4.0 cryptography==40.0.1 cycler==0.11.0 cymem==2.0.7 Cython==0.29.28 dask==2021.11.2 databricks-automl-runtime==0.2.15 databricks-cli==0.17.4 databricks-feature-store==0.10.0 dataclasses-json==0.5.7 dbl-tempo==0.1.12 dbus-python==1.2.16 debugpy==1.5.1 decorator==5.1.1 defusedxml==0.7.1 Deprecated==1.2.13 dill==0.3.4 diskcache==5.4.0 distlib==0.3.6 distributed==2021.11.2 distro==1.4.0 distro-info===0.23ubuntu1 docopt==0.6.2 docstring-to-markdown==0.11 effdet==0.3.0 effo411==0.2.0.post1 effodata==2.5.4 entrypoints==0.4 ephem==4.1.4 et-xmlfile==1.1.0 exceptiongroup==1.1.1 executing==0.8.3 facets-overview==1.0.0 fairscale==0.4.13 fastai==2.5.6 fastapi==0.95.0 fastcore==1.4.5 fastdownload==0.0.7 fastjsonschema==2.16.2 fastprogress==1.0.3 fasttext==0.9.2 filelock==3.6.0 flake8==6.0.0 Flask==1.1.2 flatbuffers==23.1.21 fonttools==4.25.0 frozenlist==1.3.3 fsspec==2022.2.0 future==0.18.2 gast==0.4.0 gdown==4.7.1 gitdb==4.0.10 GitPython==3.1.27 gluoncv==0.10.5.post0 gluonts==0.12.6 google-api-core==2.11.0 google-auth==2.17.3 google-auth-oauthlib==0.4.6 google-cloud-appengine-logging==1.3.0 google-cloud-audit-log==0.2.5 google-cloud-core==2.3.2 google-cloud-logging==3.5.0 google-cloud-storage==2.8.0 google-crc32c==1.5.0 google-pasta==0.2.0 google-resumable-media==2.4.1 googleapis-common-protos==1.59.0 gptcache==0.1.10 graphviz==0.20.1 greenlet==2.0.2 grpc-google-iam-v1==0.12.6 grpcio==1.43.0 grpcio-status==1.53.0 gunicorn==20.1.0 gviz-api==1.10.0 h11==0.14.0 h5py==3.6.0 HeapDict==1.0.1 hijri-converter==2.2.4 holidays==0.18 horovod==0.27.0 htmlmin==0.1.12 httpcore==0.16.3 httpx==0.23.3 huggingface-hub==0.12.0 humanfriendly==10.0 hyperopt==0.2.7 idna==3.3 ImageHash==4.3.1 imageio==2.27.0 imbalanced-learn==0.10.1 immutables==0.19 importlib-metadata==4.11.3 iniconfig==2.0.0 install==1.3.5 iopath==0.1.10 ipykernel==6.15.3 ipython==8.5.0 ipython-genutils==0.2.0 ipywidgets==7.7.2 isodate==0.6.1 itsdangerous==2.0.1 jedi==0.18.1 Jinja2==2.11.3 jmespath==0.10.0 joblib==1.1.1 joblibspark==0.5.1 jsonschema==4.4.0 jupyter-client==6.1.12 jupyter_core==4.11.2 jupyterlab-pygments==0.1.2 jupyterlab-widgets==1.0.0 kayday==2.1.1 keras==2.11.0 kiwisolver==1.3.2 korean-lunar-calendar==0.3.1 langchain==0.0.139 langcodes==3.3.0 layoutparser==0.3.4 libclang==15.0.6.1 lightgbm==3.3.4 llama-index==0.5.1 llvmlite==0.39.1 locket==1.0.0 LunarCalendar==0.0.9 lxml==4.9.2 Mako==1.2.0 Markdown==3.3.4 MarkupSafe==2.0.1 marshmallow==3.19.0 marshmallow-enum==1.5.1 matplotlib==3.5.1 matplotlib-inline==0.1.2 mccabe==0.7.0 mistune==0.8.4 mleap==0.20.0 mlflow-skinny==2.1.1 monotonic==1.6 mpmath==1.3.0 msg-parser==1.2.0 msgpack==1.0.5 multidict==6.0.4 multimethod==1.9.1 murmurhash==1.0.9 mypy-extensions==0.4.3 nbclient==0.5.13 nbconvert==6.4.4 nbformat==5.3.0 nest-asyncio==1.5.5 networkx==2.7.1 nlpaug==1.1.11 nltk==3.7 nodeenv==1.7.0 notebook==6.4.8 nptyping==1.4.4 num2words==0.5.12 numba==0.56.4 number-parser==0.3.0 numpy==1.22.4 oauthlib==3.2.0 olefile==0.46 omegaconf==2.3.0 onnxruntime==1.14.1 openai==0.27.4 openapi-schema-pydantic==1.2.4 opencv-python==4.6.0.66 openpyxl==3.1.2 opt-einsum==3.3.0 packaging==21.3 pandas==1.4.2 pandas-profiling==3.6.2 pandocfilters==1.5.0 paramiko==2.9.2 parso==0.8.3 partd==1.4.0 pathspec==0.9.0 pathy==0.10.1 patsy==0.5.2 pdf2image==1.16.3 pdfminer.six==20221105 pdfplumber==0.9.0 petastorm==0.12.1 pexpect==4.8.0 phik==0.12.3 pickleshare==0.7.5 Pillow==9.5.0 piqdata==2.1.20 platformdirs==2.6.2 plotly==5.6.0 pluggy==1.0.0 pmdarima==1.8.5 poirot==2.6.4 portalocker==2.7.0 preshed==3.0.8 prompt-toolkit==3.0.20 prophet==1.1.1 proto-plus==1.22.2 protobuf==3.19.6 psutil==5.8.0 psycopg2==2.9.3 ptyprocess==0.7.0 pure-eval==0.2.2 py4j==0.10.9.7 pyarrow==7.0.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pybind11==2.10.3 pycocotools==2.0.6 pycodestyle==2.10.0 pycparser==2.21 pydantic==1.10.2 pyDeprecate==0.3.2 pyflakes==3.0.1 Pygments==2.11.2 PyGObject==3.36.0 PyJWT==2.6.0 pyluach==2.2.0 PyMeeus==0.5.12 PyNaCl==1.5.0 pyodbc==4.0.32 pypandoc==1.11 pyparsing==3.0.4 PyPDF2==3.0.1 pyright==1.1.283 pyrsistent==0.18.0 PySocks==1.7.1 pyspark==3.4.0 pytesseract==0.3.10 pytest==7.3.0 pytest-mock==3.10.0 python-apt==2.0.1 python-dateutil==2.8.2 python-docx==0.8.11 python-editor==1.0.4 python-lsp-jsonrpc==1.0.0 python-lsp-server==1.6.0 python-magic==0.4.15 python-multipart==0.0.6 python-pptx==0.6.21 pytorch-lightning==1.6.5 pytorch-metric-learning==1.3.2 pytz==2021.3 PyWavelets==1.3.0 PyYAML==6.0 pyzmq==22.3.0 ray==1.13.0 regex==2022.3.15 requests==2.27.1 requests-oauthlib==1.3.1 requests-unixsocket==0.2.0 rfc3986==1.5.0 rich==13.0.1 rope==0.22.0 rsa==4.7.2 s3transfer==0.5.0 sacrebleu==2.3.1 sacremoses==0.0.53 scikit-image==0.19.3 scikit-learn==1.0.2 scipy==1.7.3 seaborn==0.11.2 Send2Trash==1.8.0 sentence-transformers==2.2.2 sentencepiece==0.1.95 setuptools-git==1.2 shap==0.41.0 simplejson==3.17.6 six==1.16.0 sktime==0.17.1 slicer==0.0.7 smart-open==5.2.1 smmap==5.0.0 sniffio==1.3.0 sortedcontainers==2.4.0 soupsieve==2.3.1 spacy==3.4.4 spacy-legacy==3.0.12 spacy-loggers==1.0.4 spark-tensorflow-distributor==1.0.0 SQLAlchemy==1.4.47 sqlparse==0.4.2 srsly==2.4.5 ssh-import-id==5.10 stack-data==0.2.0 starlette==0.26.1 statsmodels==0.13.2 sympy==1.11.1 tabulate==0.8.9 tangled-up-in-unicode==0.2.0 tbats==1.1.2 tblib==1.7.0 tenacity==8.2.2 tensorboard==2.11.2 tensorboard-data-server==0.6.1 tensorboard-plugin-profile==2.11.1 tensorboard-plugin-wit==1.8.1 tensorboardX==2.6 tensorflow==2.11.0 tensorflow-estimator==2.11.0 tensorflow-io-gcs-filesystem==0.30.0 termcolor==2.2.0 terminado==0.13.1 testpath==0.5.0 thinc==8.1.7 threadpoolctl==2.2.0 tifffile==2023.4.12 tiktoken==0.3.3 timm==0.6.13 tokenize-rt==4.2.1 tokenizers==0.12.1 tomli==1.2.2 toolz==0.12.0 torch==1.11.0 torchmetrics==0.7.3 torchvision==0.12.0 tornado==6.1 tqdm==4.64.0 traitlets==5.1.1 transformers==4.20.1 typeguard==2.13.3 typer==0.7.0 typing-inspect==0.8.0 typing_extensions==4.1.1 typish==1.9.3 tzlocal==2.1 ujson==5.1.0 unattended-upgrades==0.1 unstructured==0.5.12 unstructured-inference==0.3.2 urllib3==1.26.9 uvicorn==0.21.1 virtualenv==20.8.0 visions==0.7.5 Wand==0.6.11 wasabi==0.10.1 wcwidth==0.2.5 webencodings==0.5.1 websocket-client==0.58.0 Werkzeug==2.0.3 whatthepatch==1.0.4 widgetsnbextension==3.6.1 wrapt==1.14.1 xgboost==1.4.2 XlsxWriter==3.1.0 yacs==0.1.8 yapf==0.31.0 yarl==1.8.2 zict==2.2.0 zipp==3.7.0

[nazkhan-8451]

Also sometimes this error comes in RuntimeError: Failed to import transformers.models.gpt_neox.modeling_gpt_neox because of the following error (look up to see its traceback): 'NoneType' object has no attribute 'extensions_by_name'

[srowen]

You're using an older version of torch than I think anyone working on this does. Try 1.13? I am not sure if that's the issue, I haven't seen that error before. If that's the issue, we should pin torch>=1.13. Are you installing the requirements.txt from the repo though?

[nazkhan-8451]

Yah following the requirement.txt:

Is type of GPU cluster an issue?

[nazkhan-8451]

Alright, it's working. Apparently there was some pre-installed libraries that was giving error. Also I suggest the following way to generate text or it give CUDA NO MEMORY error

with torch.no_grad():
    print(generate_text("Explain to me the difference between nuclear fission and fusion."))
    torch.cuda.empty_cache()