environment.yml

[fireSunset]

conda env export --no-builds > environment.yml "Could you provide an environment.yml file with no-build options? I want to run it in a Windows environment. Thanks =3="

[bzhangj13zzz]

Hi, here's the yaml file. I've not tried creating environments with this before, please let me know if it works for you.

name: edc
channels:
  - pytorch
  - nvidia
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1
  - _openmp_mutex=4.5
  - abseil-cpp=20211102.0
  - aiohttp=3.9.3
  - aiosignal=1.2.0
  - arrow-cpp=14.0.2
  - asttokens=2.4.1
  - async-timeout=4.0.3
  - attrs=23.1.0
  - aws-c-auth=0.6.19
  - aws-c-cal=0.5.20
  - aws-c-common=0.8.5
  - aws-c-compression=0.2.16
  - aws-c-event-stream=0.2.15
  - aws-c-http=0.6.25
  - aws-c-io=0.13.10
  - aws-c-mqtt=0.7.13
  - aws-c-s3=0.1.51
  - aws-c-sdkutils=0.1.6
  - aws-checksums=0.1.13
  - aws-crt-cpp=0.18.16
  - aws-sdk-cpp=1.10.55
  - blas=1.0
  - boost-cpp=1.82.0
  - bottleneck=1.3.7
  - brotli-python=1.0.9
  - bzip2=1.0.8
  - c-ares=1.19.1
  - ca-certificates=2024.3.11
  - certifi=2024.2.2
  - charset-normalizer=2.0.4
  - comm=0.2.2
  - cuda-cudart=12.1.105
  - cuda-cupti=12.1.105
  - cuda-libraries=12.1.0
  - cuda-nvrtc=12.1.105
  - cuda-nvtx=12.1.105
  - cuda-opencl=12.4.127
  - cuda-runtime=12.1.0
  - datasets=2.12.0
  - debugpy=1.6.7
  - decorator=5.1.1
  - dill=0.3.6
  - exceptiongroup=1.2.0
  - executing=2.0.1
  - ffmpeg=4.3
  - filelock=3.13.1
  - freetype=2.12.1
  - frozenlist=1.4.0
  - fsspec=2023.10.0
  - gflags=2.2.2
  - glog=0.5.0
  - gmp=6.2.1
  - gmpy2=2.1.2
  - gnutls=3.6.15
  - grpc-cpp=1.48.2
  - icu=73.1
  - idna=3.4
  - importlib-metadata=7.1.0
  - importlib_metadata=7.1.0
  - intel-openmp=2023.1.0
  - ipykernel=6.29.3
  - ipython=8.18.1
  - jedi=0.19.1
  - jinja2=3.1.3
  - jpeg=9e
  - jupyter_client=8.6.1
  - jupyter_core=5.7.2
  - krb5=1.20.1
  - lame=3.100
  - lcms2=2.12
  - ld_impl_linux-64=2.38
  - lerc=3.0
  - libboost=1.82.0
  - libbrotlicommon=1.0.9
  - libbrotlidec=1.0.9
  - libbrotlienc=1.0.9
  - libcublas=12.1.0.26
  - libcufft=11.0.2.4
  - libcufile=1.9.1.3
  - libcurand=10.3.5.147
  - libcurl=8.5.0
  - libcusolver=11.4.4.55
  - libcusparse=12.0.2.55
  - libdeflate=1.17
  - libedit=3.1.20230828
  - libev=4.33
  - libevent=2.1.12
  - libffi=3.4.4
  - libgcc-ng=13.2.0
  - libgfortran-ng=11.2.0
  - libgfortran5=11.2.0
  - libgomp=13.2.0
  - libiconv=1.16
  - libidn2=2.3.4
  - libjpeg-turbo=2.0.0
  - libnghttp2=1.57.0
  - libnpp=12.0.2.50
  - libnvjitlink=12.1.105
  - libnvjpeg=12.1.1.14
  - libopenblas=0.3.21
  - libpng=1.6.39
  - libprotobuf=3.20.3
  - libsodium=1.0.18
  - libssh2=1.10.0
  - libstdcxx-ng=11.2.0
  - libtasn1=4.19.0
  - libthrift=0.15.0
  - libtiff=4.5.1
  - libunistring=0.9.10
  - libwebp-base=1.3.2
  - llvm-openmp=14.0.6
  - lz4-c=1.9.4
  - markupsafe=2.1.3
  - matplotlib-inline=0.1.6
  - mkl=2023.1.0
  - mkl-service=2.4.0
  - mkl_fft=1.3.8
  - mkl_random=1.2.4
  - mpc=1.1.0
  - mpfr=4.0.2
  - mpmath=1.3.0
  - multidict=6.0.4
  - multiprocess=0.70.14
  - ncurses=6.4
  - nest-asyncio=1.6.0
  - nettle=3.7.3
  - networkx=3.1
  - numexpr=2.8.7
  - numpy=1.26.4
  - numpy-base=1.26.4
  - openh264=2.1.1
  - openjpeg=2.4.0
  - openssl=3.2.1
  - orc=1.7.4
  - packaging=23.2
  - pandas=2.2.1
  - parso=0.8.4
  - pexpect=4.9.0
  - pickleshare=0.7.5
  - pillow=10.2.0
  - pip=23.3.1
  - platformdirs=4.2.0
  - prompt-toolkit=3.0.42
  - psutil=5.9.8
  - ptyprocess=0.7.0
  - pure_eval=0.2.2
  - pyarrow=14.0.2
  - pygments=2.17.2
  - pysocks=1.7.1
  - python=3.9.19
  - python-dateutil=2.8.2
  - python-tzdata=2023.3
  - python-xxhash=2.0.2
  - python_abi=3.9
  - pytorch=2.2.2
  - pytorch-cuda=12.1
  - pytorch-mutex=1.0
  - pytz=2023.3.post1
  - pyyaml=6.0.1
  - pyzmq=25.1.2
  - re2=2022.04.01
  - readline=8.2
  - regex=2023.10.3
  - requests=2.31.0
  - responses=0.13.3
  - s2n=1.3.27
  - safetensors=0.4.2
  - setuptools=68.2.2
  - six=1.16.0
  - snappy=1.1.10
  - sqlite=3.41.2
  - stack_data=0.6.2
  - sympy=1.12
  - tbb=2021.8.0
  - tk=8.6.12
  - torchaudio=2.2.2
  - torchtriton=2.2.0
  - torchvision=0.17.2
  - tornado=6.4
  - tqdm=4.65.0
  - traitlets=5.14.2
  - transformers=4.39.3
  - typing-extensions=4.9.0
  - typing_extensions=4.9.0
  - tzdata=2024a
  - urllib3=2.1.0
  - utf8proc=2.6.1
  - wcwidth=0.2.13
  - wheel=0.41.2
  - xxhash=0.8.0
  - xz=5.4.6
  - yaml=0.2.5
  - yarl=1.9.3
  - zeromq=4.3.5
  - zipp=3.17.0
  - zlib=1.2.13
  - zstd=1.5.5
  - pip:
      - accelerate==0.29.1
      - annotated-types==0.6.0
      - anyio==4.3.0
      - beautifulsoup4==4.12.3
      - blinker==1.8.2
      - bs4==0.0.2
      - click==8.1.7
      - dash==2.17.0
      - dash-core-components==2.0.0
      - dash-html-components==2.0.0
      - dash-table==5.0.0
      - distro==1.9.0
      - evaluate==0.4.1
      - flask==3.0.3
      - h11==0.14.0
      - httpcore==1.0.5
      - httpx==0.27.0
      - huggingface-hub==0.23.2
      - itsdangerous==2.2.0
      - joblib==1.4.2
      - lxml==5.2.2
      - nervaluate==0.1.8
      - nltk==3.8.1
      - openai==1.16.2
      - peft==0.10.0
      - plotly==5.22.0
      - protobuf==5.27.0
      - pydantic==2.6.4
      - pydantic-core==2.16.3
      - retrying==1.3.4
      - scikit-learn==1.5.0
      - scipy==1.13.1
      - sentencepiece==0.2.0
      - sniffio==1.3.1
      - soupsieve==2.5
      - tenacity==8.3.0
      - threadpoolctl==3.5.0
      - tokenizers==0.15.2
      - unidecode==1.3.8
      - werkzeug==3.0.3

[fireSunset]

I am glad to receive your reply. I successfully set up the environment yesterday according to the installation package provided by pip, but when running, I encountered the error shown in the image. I am not sure if it is an issue with the environment.😭

[bzhangj13zzz]

This might be an issue with the transformers library. I am not sure if it is Windows-specific. Can you share with me the versions of transformers and tokenizers that you are using? I will see if I can reproduce the errors on Linux as I currently do not have a Windows machine to test with. Thanks.

[fireSunset]

I have set up the version to match the environment provided, but to prevent issues caused by environment installation, I started running the code from a Linux environment on the server, and I am still in the process of downloading the model. thanks =3=

[fireSunset]

I tried to run the code in a Linux environment and encountered the same error.

[bzhangj13zzz]

It seems to be due to an update on the loading of tokenizers, https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1/discussions/229

Updating the transformers package should fix it. pip install -U transformers

I will update the environment yaml file later.

Please let me know if the error still persists.

[bzhangj13zzz]

I am not exactly sure why updates of packages will break old code.. perhaps because loading of the tokenizers and models uses some remote files which are updated and no longer compatible with the old packages.

[fireSunset]

咱们好像都是中国人嘞哈哈哈，我直接用中文吧，新传上来的环境文件直接冲突了

[bzhangj13zzz]

有点奇怪，我试了两台server都没问题，请问你的conda版本是什么？可以用conda -V查看

[fireSunset]

[bzhangj13zzz]

可能是conda版本的问题，https://github.com/conda/conda/issues/12155#issuecomment-1341608573， 你可以参考一下这个issue，更新一下conda试一下

conda update --name base --override-channels --channel defaults --all

[fireSunset]

我更新了transformer版本，可以运行了现在，我大概改了这些