AnswerDotAI / cold-compress

Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of GPT-Fast, a simple, PyTorch-native generation codebase.
https://www.answer.ai/posts/2024-08-01-cold-compress.html
BSD 3-Clause "New" or "Revised" License
86 stars 8 forks source link

torch dependency results in error #41

Open maxjeblick opened 3 months ago

maxjeblick commented 3 months ago

I'm getting the error below when running any torch code. This is probably due to an incompatible cuda version (requirements.txt specifies cu121).

I would suggest to

(venv)  max@max  ~/PycharmProjects/cold-compress   main  python3 -mtorch.utils.collect_env
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 112, in _get_module_details
  File "/home/max/PycharmProjects/cold-compress/venv/lib/python3.11/site-packages/torch/__init__.py", line 364, in <module>
    _load_global_deps()
  File "/home/max/PycharmProjects/cold-compress/venv/lib/python3.11/site-packages/torch/__init__.py", line 320, in _load_global_deps
    raise err
  File "/home/max/PycharmProjects/cold-compress/venv/lib/python3.11/site-packages/torch/__init__.py", line 298, in _load_global_deps
    ctypes.CDLL(global_deps_lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/usr/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /home/max/PycharmProjects/cold-compress/venv/lib/python3.11/site-packages/torch/lib/libtorch_global_deps.so: cannot open shared object file: No such file or directory
griff4692 commented 3 months ago

Thanks for flagging this.

We can unpin the cuda requirement and test internally with others.

dev20240723 was chosen because a bugfix was made by the PyTorch team that allowed our code to compile without error.

For now, feel free to try by installing the latest PyTorch nightly with your specific CUDA version.

We will do some testing internally and update the README.