search

mlabonne / llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
https://mlabonne.github.io/blog/
Apache License 2.0
39.18k stars 4.14k forks source link

Error when trying to quantize GGUF. #64

Open Samdoses opened 7 months ago

Samdoses commented 7 months ago

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. pandas-stubs 2.0.3.230814 requires numpy>=1.25.0; python_version >= "3.9", but you have numpy 1.24.4 which is incompatible. tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 4.25.3 which is incompatible. torchaudio 2.2.1+cu121 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible. torchtext 0.17.1 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible. torchvision 0.17.1+cu121 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible. Successfully installed einops-0.7.0 gguf-0.6.0 numpy-1.24.4 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.18.1 nvidia-nvjitlink-cu12-12.4.127 nvidia-nvtx-cu12-12.1.105 protobuf-4.25.3 torch-2.1.2 triton-2.1.0 WARNING: The following packages were previously imported in this runtime: [numpy] You must restart the runtime in order to use newly installed versions.

Samdoses commented 7 months ago

I understand that the wrong version of several dependencies are being installed from the requirements.txt of llama.cpp. Is there any way to manually set the misbehaving dependencie's version myself in the collab code?

Thanks!

mlabonne commented 7 months ago

Yes, you can add a line after the installation of the requirements:

!pip install your-library

On Sat, Apr 20, 2024, 17:39 Samdoses @.***> wrote:

I understand that the wrong version of several dependancies are being installed from the requirements.txt of llama.cpp. Is there any way to manually set the dependancy version myself in the collab code?

— Reply to this email directly, view it on GitHub https://github.com/mlabonne/llm-course/issues/64#issuecomment-2067724807, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATL5EGS4AOHWEOI37WJH6BLY6KK33AVCNFSM6AAAAABGQTW3QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXG4ZDIOBQG4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Samdoses commented 7 months ago

I tried installing the dependencies, but they seem to be uninstalled immediately. Is there a way to prevent this and force the code to use the updated versions instead of those specified in the requirements file?

Attempting uninstall: protobuf Found existing installation: protobuf 3.20.3 Uninstalling protobuf-3.20.3: Successfully uninstalled protobuf-3.20.3 Attempting uninstall: numpy Found existing installation: numpy 1.25.2 Uninstalling numpy-1.25.2: Successfully uninstalled numpy-1.25.2 Attempting uninstall: torch Found existing installation: torch 2.2.1+cu121 Uninstalling torch-2.2.1+cu121: Successfully uninstalled torch-2.2.1+cu121

jazelly commented 6 months ago

Hi, have you considered using venv or conda? Looks like you got some existing deps that are unhappy with the requirements.txt that you are trying to install