allenai / allennlp

An open-source NLP research library, built on PyTorch.
http://www.allennlp.org
Apache License 2.0
11.71k stars 2.24k forks source link

file link error on Google Colaboratory #5696

Closed hongyi-li-93 closed 1 year ago

hongyi-li-93 commented 1 year ago

Checklist

Description

When training on Google Colab, hard copying file fails. (I guess adding 38 to line 620 of file_utils.py might help.)

Python traceback:

``` [/usr/local/lib/python3.7/dist-packages/allennlp/common/file_utils.py](https://localhost:8080/#) in hardlink_or_copy(source, dest) 616 def hardlink_or_copy(source: PathOrStr, dest: PathOrStr): 617 try: --> 618 os.link(source, dest) 619 except OSError as e: 620 if e.errno in {18, 95}: # Cross-device link and Windows OSError: [Errno 38] Function not implemented: 'source.txt' -> 'dest.txt' ```

Related issues or possible duplicates

Environment

OS: Linux(Google Colaboratory)

Python version: 3.7.13

Output of pip freeze:

``` absl-py==1.2.0 aiohttp==3.8.1 aiosignal==1.2.0 alabaster==0.7.12 albumentations==0.1.12 allennlp==2.10.0 altair==4.2.0 appdirs==1.4.4 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 arviz==0.12.1 astor==0.8.1 astropy==4.3.1 astunparse==1.6.3 async-timeout==4.0.2 asynctest==0.13.0 atari-py==0.2.9 atomicwrites==1.4.1 attrs==21.4.0 audioread==2.1.9 autograd==1.4 Babel==2.10.3 backcall==0.2.0 base58==2.1.1 beautifulsoup4==4.6.3 bleach==5.0.1 blis==0.7.8 bokeh==2.3.3 boto3==1.24.42 botocore==1.27.42 branca==0.5.0 bs4==0.0.1 CacheControl==0.12.11 cached-path==1.1.5 cached-property==1.5.2 cachetools==4.2.4 catalogue==2.0.8 certifi==2022.6.15 cffi==1.15.1 cftime==1.6.1 chardet==3.0.4 charset-normalizer==2.1.0 click==7.1.2 clikit==0.6.2 cloudpickle==1.3.0 cmake==3.22.5 cmdstanpy==1.0.4 colorcet==3.0.0 colorlover==0.3.0 commonmark==0.9.1 community==1.0.0b1 contextlib2==0.5.5 convertdate==2.4.0 coverage==3.7.1 coveralls==0.5 crashtest==0.3.1 crcmod==1.7 cufflinks==0.17.3 cvxopt==1.2.7 cvxpy==1.0.31 cycler==0.11.0 cymem==2.0.6 Cython==0.29.30 daft==0.0.4 dask==2.12.0 datascience==0.10.6 debugpy==1.0.0 decorator==4.4.2 defusedxml==0.7.1 deprecat==2.1.1 descartes==1.1.0 dill==0.3.5.1 distributed==1.25.3 dlib==19.24.0 dm-tree==0.1.7 docker-pycreds==0.4.0 docopt==0.6.2 docutils==0.17.1 dopamine-rl==1.0.5 earthengine-api==0.1.317 easydict==1.9 ecos==2.0.10 editdistance==0.5.3 en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.4.0/en_core_web_sm-3.4.0-py3-none-any.whl entrypoints==0.4 ephem==4.1.3 et-xmlfile==1.1.0 etils==0.6.0 fa2==0.3.5 fairscale==0.4.6 fastai==2.7.7 fastcore==1.5.10 fastdownload==0.0.7 fastdtw==0.3.4 fastjsonschema==2.16.1 fastprogress==1.0.3 fastrlock==0.8 feather-format==0.4.1 filelock==3.7.1 firebase-admin==4.4.0 fix-yahoo-finance==0.0.22 Flask==1.1.4 flatbuffers==2.0 folium==0.8.3 frozenlist==1.3.0 future==0.16.0 gast==0.5.3 GDAL==2.2.2 gdown==4.4.0 gensim==3.6.0 geographiclib==1.52 geopy==1.17.0 gin-config==0.5.0 gitdb==4.0.9 GitPython==3.1.27 glob2==0.7 google==2.0.3 google-api-core==1.31.6 google-api-python-client==1.12.11 google-auth==1.35.0 google-auth-httplib2==0.0.4 google-auth-oauthlib==0.4.6 google-cloud-bigquery==1.21.0 google-cloud-bigquery-storage==1.1.2 google-cloud-core==2.3.2 google-cloud-datastore==1.8.0 google-cloud-firestore==1.7.0 google-cloud-language==1.2.0 google-cloud-storage==2.4.0 google-cloud-translate==1.5.0 google-colab @ file:///colabtools/dist/google-colab-1.0.0.tar.gz google-crc32c==1.3.0 google-pasta==0.2.0 google-resumable-media==2.3.3 googleapis-common-protos==1.56.4 googledrivedownloader==0.4 graphviz==0.10.1 greenlet==1.1.2 grpcio==1.47.0 gspread==3.4.2 gspread-dataframe==3.0.8 gym==0.17.3 h5py==3.7.0 HeapDict==1.0.1 hijri-converter==2.2.4 holidays==0.14.2 holoviews==1.14.9 html5lib==1.0.1 httpimport==0.5.18 httplib2==0.17.4 httplib2shim==0.0.3 httpstan==4.6.1 huggingface-hub==0.8.1 humanize==0.5.1 hyperopt==0.1.2 idna==2.10 imageio==2.4.1 imagesize==1.4.1 imbalanced-learn==0.8.1 imblearn==0.0 imgaug==0.2.9 importlib-metadata==4.12.0 importlib-resources==5.9.0 imutils==0.5.4 inflect==2.1.0 iniconfig==1.1.1 intel-openmp==2022.1.0 intervaltree==2.1.0 ipykernel==4.10.1 ipython==5.5.0 ipython-genutils==0.2.0 ipython-sql==0.3.9 ipywidgets==7.7.1 itsdangerous==1.1.0 jax==0.3.14 jaxlib @ https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.3.14+cuda11.cudnn805-cp37-none-manylinux2014_x86_64.whl jedi==0.18.1 jieba==0.42.1 Jinja2==2.11.3 jmespath==1.0.1 joblib==1.1.0 jpeg4py==0.1.4 jsonnet==0.18.0 jsonschema==4.3.3 jupyter==1.0.0 jupyter-client==5.3.5 jupyter-console==5.2.0 jupyter-core==4.11.1 jupyterlab-pygments==0.2.2 jupyterlab-widgets==1.1.1 kaggle==1.5.12 kapre==0.3.7 keras==2.8.0 Keras-Preprocessing==1.1.2 keras-vis==0.4.1 kiwisolver==1.4.4 korean-lunar-calendar==0.2.1 langcodes==3.3.0 libclang==14.0.1 librosa==0.8.1 lightgbm==2.2.3 llvmlite==0.34.0 lmdb==1.3.0 LunarCalendar==0.0.9 lxml==4.2.6 Markdown==3.4.1 MarkupSafe==2.0.1 marshmallow==3.17.0 matplotlib==3.2.2 matplotlib-inline==0.1.3 matplotlib-venn==0.11.7 missingno==0.5.1 mistune==0.8.4 mizani==0.6.0 mkl==2019.0 mlxtend==0.14.0 more-itertools==8.13.0 moviepy==0.2.3.5 mpmath==1.2.1 msgpack==1.0.4 multidict==6.0.2 multiprocess==0.70.13 multitasking==0.0.11 murmurhash==1.0.7 music21==5.5.0 natsort==5.5.0 nbclient==0.6.6 nbconvert==5.6.1 nbformat==5.4.0 nest-asyncio==1.5.5 netCDF4==1.6.0 networkx==2.6.3 nibabel==3.0.2 nltk==3.7 notebook==5.3.1 numba==0.51.2 numexpr==2.8.3 numpy==1.21.6 oauth2client==4.1.3 oauthlib==3.2.0 okgrade==0.4.3 opencv-contrib-python==4.6.0.66 opencv-python==4.6.0.66 openpyxl==3.0.10 opt-einsum==3.3.0 osqp==0.6.2.post0 overrides==6.1.0 packaging==21.3 palettable==3.3.0 pandas==1.3.5 pandas-datareader==0.9.0 pandas-gbq==0.13.3 pandas-profiling==1.4.1 pandocfilters==1.5.0 panel==0.12.1 param==1.12.2 parso==0.8.3 pastel==0.2.1 pathlib==1.0.1 pathtools==0.1.2 pathy==0.6.2 patsy==0.5.2 pep517==0.12.0 pexpect==4.8.0 pickleshare==0.7.5 Pillow==7.1.2 pip-tools==6.2.0 plotly==5.5.0 plotnine==0.6.0 pluggy==1.0.0 pooch==1.6.0 portpicker==1.3.9 prefetch-generator==1.0.1 preshed==3.0.6 prettytable==3.3.0 progressbar2==3.38.0 prometheus-client==0.14.1 promise==2.3 prompt-toolkit==1.0.18 prophet==1.1 protobuf==3.20.0 psutil==5.4.8 psycopg2==2.7.6.1 ptyprocess==0.7.0 py==1.11.0 pyarrow==6.0.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycocotools==2.0.4 pycparser==2.21 pyct==0.4.8 pydantic==1.8.2 pydata-google-auth==1.4.0 pydot==1.3.0 pydot-ng==2.0.0 pydotplus==2.0.2 PyDrive==1.3.1 pyemd==0.5.1 pyerfa==2.0.0.1 pyglet==1.5.0 Pygments==2.6.1 pygobject==3.26.1 pylev==1.4.0 pymc3==3.11.5 PyMeeus==0.5.11 pymongo==4.2.0 pymystem3==0.2.0 PyOpenGL==3.1.6 pyparsing==3.0.9 pyrsistent==0.18.1 pysimdjson==3.2.0 pysndfile==1.3.8 PySocks==1.7.1 pystan==3.3.0 pytest==7.1.2 python-apt==0.0.0 python-chess==0.23.11 python-dateutil==2.8.2 python-louvain==0.16 python-slugify==6.1.2 python-utils==3.3.3 pytorch-pretrained-bert==0.6.2 pytorch-transformers==1.2.0 pytz==2022.1 pyviz-comms==2.2.0 PyWavelets==1.3.0 PyYAML==6.0 pyzmq==23.2.0 qdldl==0.1.5.post2 qtconsole==5.3.1 QtPy==2.1.0 regex==2022.6.2 requests==2.28.1 requests-oauthlib==1.3.1 resampy==0.3.1 rich==12.1.0 rpy2==3.4.5 rsa==4.9 s3transfer==0.6.0 sacremoses==0.0.53 scikit-image==0.18.3 scikit-learn==1.0.2 scipy==1.7.3 screen-resolution-extra==0.0.0 scs==3.2.0 seaborn==0.11.2 semver==2.13.0 Send2Trash==1.8.0 sentencepiece==0.1.96 sentry-sdk==1.9.0 setproctitle==1.3.0 setuptools-git==1.2 Shapely==1.8.2 shortuuid==1.0.9 simplegeneric==0.8.1 six==1.15.0 sklearn==0.0 sklearn-pandas==1.8.0 smart-open==5.2.1 smmap==5.0.0 snowballstemmer==2.2.0 sortedcontainers==2.4.0 SoundFile==0.10.3.post1 soupsieve==2.3.2.post1 spacy==3.3.1 spacy-legacy==3.0.9 spacy-loggers==1.0.3 Sphinx==1.8.6 sphinxcontrib-serializinghtml==1.1.5 sphinxcontrib-websupport==1.2.4 SQLAlchemy==1.4.39 sqlparse==0.4.2 srsly==2.4.4 statsmodels==0.10.2 sympy==1.7.1 tables==3.7.0 tabulate==0.8.10 tblib==1.7.0 tenacity==8.0.1 tensorboard==2.8.0 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tensorboardX==2.5.1 tensorflow==2.8.2+zzzcolab20220719082949 tensorflow-datasets==4.0.1 tensorflow-estimator==2.8.0 tensorflow-gcs-config==2.8.0 tensorflow-hub==0.12.0 tensorflow-io-gcs-filesystem==0.26.0 tensorflow-metadata==1.9.0 tensorflow-probability==0.16.0 termcolor==1.1.0 terminado==0.13.3 testpath==0.6.0 text-unidecode==1.3 textblob==0.15.3 Theano-PyMC==1.1.2 thinc==8.0.17 threadpoolctl==3.1.0 tifffile==2021.11.2 tinycss2==1.1.1 tokenizers==0.12.1 tomli==2.0.1 toolz==0.12.0 torch==1.11.0 torchaudio @ https://download.pytorch.org/whl/cu113/torchaudio-0.12.0%2Bcu113-cp37-cp37m-linux_x86_64.whl torchsummary==1.5.1 torchtext==0.13.0 torchvision==0.12.0 tornado==5.1.1 tqdm==4.64.0 traitlets==5.3.0 transformers==4.20.1 tweepy==3.10.0 typeguard==2.7.1 typer==0.4.2 typing-extensions==4.1.1 typing-utils==0.1.0 tzlocal==1.5.1 ujson==5.4.0 uritemplate==3.0.1 urllib3==1.26.11 vega-datasets==0.9.0 wandb==0.12.21 wasabi==0.9.1 wcwidth==0.2.5 webargs==8.2.0 webencodings==0.5.1 Werkzeug==1.0.1 widgetsnbextension==3.6.1 wordcloud==1.5.0 wrapt==1.14.1 xarray==0.20.2 xarray-einstats==0.2.2 xgboost==0.90 xkit==0.0.0 xlrd==1.1.0 xlwt==1.3.0 yarl==1.7.2 yellowbrick==1.4 zict==2.2.0 zipp==3.8.1 ```

Steps to reproduce

Example source:

``` import allennlp s = 'source.txt' open(s,'a+') d = 'dest.txt' allennlp.common.file_utils.hardlink_or_copy(s, d) ```

github-actions[bot] commented 1 year ago

This issue is being closed due to lack of activity. If you think it still needs to be addressed, please comment on this thread 👇