Closed Treide1 closed 1 month ago
P.S.:
I use the pre-built llama-cpp-python wheel on Windows 10:
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
For clarity, after installing torch and transformers, I run into this:
OSError: [WinError 127] Die angegebene Prozedur wurde nicht gefunden. Error loading "c:\git\outlines_playground\.venv\Lib\site-packages\torch\lib\torch_python.dll" or one of its dependencies.
It sounds like a torch
install issue to me. Are you able to install and import torch
independently from Outlines?
@Treide1 could you please try pip install git+https://github.com/outlines-dev/outlines
?
outlines.generate.json
no longer uses outlines.integrations.llamacpp
in main
Thanks for your advice!
Installing from current main
resolved the issue 👍
Here my steps, for completeness:
pip install transformers datasets accelerate torch
(even though i didn't intend on using transformers).llama-cpp-python
using a pre-built wheel as before and ran one of their examples. Inference works here.pip install git+https://github.com/outlines-dev/outlines
as recommended.Llama.__del__
still throws on terminating program).character = generator("Give me a character description", max_tokens=100)
print(f"Generated character: {character}")
# Output:
# Generated character: name='Kael' age=2862
In conclusion, the pip installation of outlines as-is was broken for me.
Describe the issue as clearly as possible:
After a clean installation in a .venv (pydantic, llama-cpp-python, outlines), running your example for llama_cpp I get an error that on
import torch
that yieldsModuleNotFoundError: No module named 'torch'
The llama_cpp installation worked. I can their examples just fine.
After some testing, I see similar effects on missing lib
transformers
. Maybe there are some more.Installing torch (
pip install torch
) leads to a cascade of incompatibilities, presumably due to bad versions.Is there a way to understand which libs in which versions are needed for the examples ? Have I missed some documentation discussing this ?
Thanks in advance.
Steps/code to reproduce the bug:
Expected result:
Error message:
Outlines/Python version information:
Context for the issue:
No response