Open krish240574 opened 4 months ago
4entertainment
commented
4 months ago @krish240574 Hello, i used the answers below to solve this problem.
Enjoy your work
matheusft
commented
1 month ago I'm running on a CPU-only host.
My pip freeze output: aiohttp==3.9.5 aiosignal==1.3.1 annotated-types==0.6.0 anyio==4.3.0 async-timeout==4.0.3 attrs==23.2.0 beautifulsoup4==4.12.3 bitarray==2.9.2 blinker==1.8.2 catalogue==2.0.10 certifi==2024.2.2 charset-normalizer==3.3.2 click==8.1.7 colbert-ai==0.2.19 dataclasses-json==0.6.6 datasets==2.19.1 Deprecated==1.2.14 dill==0.3.8 dirtyjson==1.0.8 distro==1.9.0 exceptiongroup==1.2.1 faiss-cpu==1.8.0 fast-pytorch-kmeans==0.2.0.1 filelock==3.14.0 Flask==3.0.3 frozenlist==1.4.1 fsspec==2024.3.1 git-python==1.0.3 gitdb==4.0.11 GitPython==3.1.43 greenlet==3.0.3 h11==0.14.0 httpcore==1.0.5 httpx==0.27.0 huggingface-hub==0.23.0 idna==3.7 itsdangerous==2.2.0 Jinja2==3.1.4 joblib==1.4.2 jsonpatch==1.33 jsonpointer==2.4 langchain==0.1.19 langchain-community==0.0.38 langchain-core==0.1.52 langchain-text-splitters==0.0.1 langsmith==0.1.56 llama-index==0.10.36 llama-index-agent-openai==0.2.4 llama-index-cli==0.1.12 llama-index-core==0.10.36 llama-index-embeddings-openai==0.1.9 llama-index-indices-managed-llama-cloud==0.1.6 llama-index-legacy==0.9.48 llama-index-llms-openai==0.1.18 llama-index-multi-modal-llms-openai==0.1.5 llama-index-program-openai==0.1.6 llama-index-question-gen-openai==0.1.3 llama-index-readers-file==0.1.22 llama-index-readers-llama-parse==0.1.4 llama-parse==0.4.2 llamaindex-py-client==0.1.19 MarkupSafe==2.1.5 marshmallow==3.21.2 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.16 mypy-extensions==1.0.0 nest-asyncio==1.6.0 networkx==3.3 ninja==1.11.1.1 nltk==3.8.1 numpy==1.26.4 onnx==1.16.0 openai==1.28.0 orjson==3.10.3 packaging==23.2 pandas==2.2.2 pillow==10.3.0 protobuf==5.26.1 pyarrow==16.0.0 pyarrow-hotfix==0.6 pydantic==2.7.1 pydantic_core==2.18.2 pynvml==11.5.0 pypdf==4.2.0 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 pytz==2024.1 PyYAML==6.0.1 RAGatouille==0.0.8.post2 regex==2024.5.10 requests==2.31.0 safetensors==0.4.3 scikit-learn==1.4.2 scipy==1.13.0 sentence-transformers==2.7.0 six==1.16.0 smmap==5.0.1 sniffio==1.3.1 soupsieve==2.5 SQLAlchemy==2.0.30 srsly==2.4.8 striprtf==0.0.26 sympy==1.12 tenacity==8.3.0 threadpoolctl==3.5.0 tiktoken==0.6.0 tokenizers==0.19.1 torch==2.2.0+cpu tqdm==4.66.4 transformers==4.40.2 typing-inspect==0.9.0 typing_extensions==4.11.0 tzdata==2024.1 ujson==5.9.0 urllib3==2.2.1 voyager==2.0.6 Werkzeug==3.0.3 wrapt==1.16.0 xxhash==3.4.1 yarl==1.9.4
The code in question is : def create_or_get_colbert_model(username): index_name=username+"_report_history" index_path = PATH/.ragatouille/colbert/indexes/"+index_name+"/" print("Index path = ",index_path) if os.path.exists(index_path): RAG = RAGPretrainedModel.from_index(index_path) else: RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0") return RAG
After printing "Index path = " , index_path, I get : Loading segmented_maxsim_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
This happens even after creating a fresh ENV, and fresh installs of torch-cpu-2.2.0, sentence_transformers, transformers and ragatouille.
I'm getting the same on a Apple M1 machine using Python 3.10.6
Even when running the code below
from ragatouille import RAGPretrainedModel
if __name__ == "__main__":
RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")
The code gets stuck on Loading segmented_maxsim_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
This is my env
Package Version
---------------------------------------- -----------
aiohappyeyeballs 2.3.4
aiohttp 3.10.0
aiosignal 1.3.1
annotated-types 0.7.0
anyio 4.4.0
appdirs 1.4.4
asgiref 3.8.1
async-timeout 4.0.3
attrs 23.2.0
backoff 2.2.1
bcrypt 4.2.0
beautifulsoup4 4.12.3
bitarray 2.9.2
blinker 1.8.2
build 1.2.1
cachetools 5.4.0
catalogue 2.0.10
certifi 2024.7.4
cffi 1.16.0
charset-normalizer 3.3.2
chroma-hnswlib 0.7.6
chromadb 0.5.5
click 8.1.7
colbert-ai 0.2.19
coloredlogs 15.0.1
cryptography 43.0.0
dataclasses-json 0.6.7
datasets 2.20.0
db-dtypes 1.2.0
Deprecated 1.2.14
dill 0.3.8
dirtyjson 1.0.8
distro 1.9.0
docstring_parser 0.16
exceptiongroup 1.2.2
faiss-cpu 1.8.0.post1
fast-pytorch-kmeans 0.2.0.1
fastapi 0.112.0
filelock 3.15.4
Flask 3.0.3
flatbuffers 24.3.25
frozenlist 1.4.1
fsspec 2024.5.0
git-python 1.0.3
gitdb 4.0.11
GitPython 3.1.43
google-api-core 2.19.1
google-auth 2.32.0
google-auth-oauthlib 1.2.1
google-cloud-aiplatform 1.60.0
google-cloud-bigquery 3.25.0
google-cloud-core 2.4.1
google-cloud-resource-manager 1.12.5
google-cloud-storage 2.18.0
google-crc32c 1.5.0
google-resumable-media 2.7.1
googleapis-common-protos 1.63.2
greenlet 3.0.3
grpc-google-iam-v1 0.13.1
grpcio 1.65.2
grpcio-status 1.62.2
h11 0.14.0
httpcore 1.0.5
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.24.5
humanfriendly 10.0
idna 3.7
importlib_metadata 8.0.0
importlib_resources 6.4.0
itsdangerous 2.2.0
Jinja2 3.1.4
joblib 1.4.2
jsonpatch 1.33
jsonpointer 3.0.0
kubernetes 30.1.0
langchain 0.1.20
langchain-community 0.0.38
langchain-core 0.1.52
langchain-experimental 0.0.58
langchain-google-vertexai 1.0.4
langchain-huggingface 0.0.3
langchain-openai 0.1.7
langchain-text-splitters 0.0.2
langsmith 0.1.96
llama-cloud 0.0.11
llama-index 0.10.59
llama-index-agent-openai 0.2.9
llama-index-cli 0.1.13
llama-index-core 0.10.59
llama-index-embeddings-openai 0.1.11
llama-index-indices-managed-llama-cloud 0.2.7
llama-index-legacy 0.9.48
llama-index-llms-openai 0.1.27
llama-index-multi-modal-llms-openai 0.1.8
llama-index-program-openai 0.1.7
llama-index-question-gen-openai 0.1.3
llama-index-readers-file 0.1.32
llama-index-readers-llama-parse 0.1.6
llama-parse 0.4.9
markdown-it-py 3.0.0
MarkupSafe 2.1.5
marshmallow 3.21.3
mdurl 0.1.2
mmh3 4.1.0
monotonic 1.6
mpmath 1.3.0
multidict 6.0.5
multiprocess 0.70.16
munch 4.0.0
mypy-extensions 1.0.0
nest-asyncio 1.6.0
networkx 3.3
ninja 1.11.1.1
nltk 3.8.1
numpy 1.26.4
oauthlib 3.2.2
onnx 1.16.2
onnxruntime 1.18.1
openai 1.38.0
opentelemetry-api 1.26.0
opentelemetry-exporter-otlp-proto-common 1.26.0
opentelemetry-exporter-otlp-proto-grpc 1.26.0
opentelemetry-instrumentation 0.47b0
opentelemetry-instrumentation-asgi 0.47b0
opentelemetry-instrumentation-fastapi 0.47b0
opentelemetry-proto 1.26.0
opentelemetry-sdk 1.26.0
opentelemetry-semantic-conventions 0.47b0
opentelemetry-util-http 0.47b0
orjson 3.10.6
overrides 7.7.0
packaging 23.2
pandas 2.2.2
pandas-gbq 0.23.1
pillow 10.4.0
pip 22.2.1
posthog 3.5.0
proto-plus 1.24.0
protobuf 4.25.4
pyarrow 17.0.0
pyarrow-hotfix 0.6
pyasn1 0.6.0
pyasn1_modules 0.4.0
pycparser 2.22
pydantic 2.8.2
pydantic_core 2.20.1
pydata-google-auth 1.8.2
Pygments 2.18.0
pynvml 11.5.3
pypdf 4.3.1
PyPika 0.48.9
pyproject_hooks 1.1.0
pysbd 0.3.4
python-certifi-win32 1.6.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
pytz 2024.1
PyYAML 6.0.1
ragas 0.1.12
RAGatouille 0.0.8.post2
rank-bm25 0.2.2
regex 2024.7.24
requests 2.32.3
requests-oauthlib 2.0.0
rich 13.7.1
rsa 4.9
safetensors 0.4.3
scikit-learn 1.5.1
scipy 1.14.0
sentence-transformers 2.7.0
setuptools 63.2.0
setuptools-scm 8.1.0
shapely 2.0.5
shellingham 1.5.4
simsimd 3.7.7
six 1.16.0
smmap 5.0.1
sniffio 1.3.1
soupsieve 2.5
SQLAlchemy 2.0.31
srsly 2.4.8
starlette 0.37.2
striprtf 0.0.26
sympy 1.13.1
tenacity 8.5.0
threadpoolctl 3.5.0
tiktoken 0.7.0
tokenizers 0.19.1
tomli 2.0.1
torch 2.4.0
tqdm 4.66.4
transformers 4.43.3
typer 0.12.3
typing_extensions 4.12.2
typing-inspect 0.9.0
tzdata 2024.1
ujson 5.10.0
urllib3 2.2.2
uvicorn 0.30.5
uvloop 0.19.0
vertexai 1.60.0
voyager 2.0.9
watchfiles 0.22.0
websocket-client 1.8.0
websockets 12.0
Werkzeug 3.0.3
wrapt 1.16.0
xxhash 3.4.1
yarl 1.9.4
zipp 3.19.2Right, I've found a fix that worked for me. https://github.com/zhou13/neurvps/issues/1#issuecomment-820898095
Deleting the /Users/MY_USER/Library/Caches/torch_extensions/py310_cpu/segmented_maxsim_cpp/lock file solves the issue.
If the path above does not work for you, put a breakpoint at line baton = FileBaton(os.path.join(build_directory, 'lock')) from file MY_VENV/lib/python3.10/site-packages/torch/utils/cpp_extension.py and check the value for build_directory
ColBERT uses this file when doing RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")
bclavie
commented
3 weeks ago Hey!
I've also finally debugged this, and the fix in 99% of cases I've seen is what @matheusft pointed out above: deleting the torch extensions lock and partially built binaries in the torch extensions cache. I'll try and include a clever way of auto-doing this in an upcoming version, if there is one that doesn't add too much overhead. It seems like whenever a build fails for any reason (terrible luck, keyboard interrupt, etc...), there is a chance torch never clears this, and future load assume it's already built, and get stuck trying to load "nothing" rather than erroring out.
I'm running on a CPU-only host.
My pip freeze output: aiohttp==3.9.5 aiosignal==1.3.1 annotated-types==0.6.0 anyio==4.3.0 async-timeout==4.0.3 attrs==23.2.0 beautifulsoup4==4.12.3 bitarray==2.9.2 blinker==1.8.2 catalogue==2.0.10 certifi==2024.2.2 charset-normalizer==3.3.2 click==8.1.7 colbert-ai==0.2.19 dataclasses-json==0.6.6 datasets==2.19.1 Deprecated==1.2.14 dill==0.3.8 dirtyjson==1.0.8 distro==1.9.0 exceptiongroup==1.2.1 faiss-cpu==1.8.0 fast-pytorch-kmeans==0.2.0.1 filelock==3.14.0 Flask==3.0.3 frozenlist==1.4.1 fsspec==2024.3.1 git-python==1.0.3 gitdb==4.0.11 GitPython==3.1.43 greenlet==3.0.3 h11==0.14.0 httpcore==1.0.5 httpx==0.27.0 huggingface-hub==0.23.0 idna==3.7 itsdangerous==2.2.0 Jinja2==3.1.4 joblib==1.4.2 jsonpatch==1.33 jsonpointer==2.4 langchain==0.1.19 langchain-community==0.0.38 langchain-core==0.1.52 langchain-text-splitters==0.0.1 langsmith==0.1.56 llama-index==0.10.36 llama-index-agent-openai==0.2.4 llama-index-cli==0.1.12 llama-index-core==0.10.36 llama-index-embeddings-openai==0.1.9 llama-index-indices-managed-llama-cloud==0.1.6 llama-index-legacy==0.9.48 llama-index-llms-openai==0.1.18 llama-index-multi-modal-llms-openai==0.1.5 llama-index-program-openai==0.1.6 llama-index-question-gen-openai==0.1.3 llama-index-readers-file==0.1.22 llama-index-readers-llama-parse==0.1.4 llama-parse==0.4.2 llamaindex-py-client==0.1.19 MarkupSafe==2.1.5 marshmallow==3.21.2 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.16 mypy-extensions==1.0.0 nest-asyncio==1.6.0 networkx==3.3 ninja==1.11.1.1 nltk==3.8.1 numpy==1.26.4 onnx==1.16.0 openai==1.28.0 orjson==3.10.3 packaging==23.2 pandas==2.2.2 pillow==10.3.0 protobuf==5.26.1 pyarrow==16.0.0 pyarrow-hotfix==0.6 pydantic==2.7.1 pydantic_core==2.18.2 pynvml==11.5.0 pypdf==4.2.0 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 pytz==2024.1 PyYAML==6.0.1 RAGatouille==0.0.8.post2 regex==2024.5.10 requests==2.31.0 safetensors==0.4.3 scikit-learn==1.4.2 scipy==1.13.0 sentence-transformers==2.7.0 six==1.16.0 smmap==5.0.1 sniffio==1.3.1 soupsieve==2.5 SQLAlchemy==2.0.30 srsly==2.4.8 striprtf==0.0.26 sympy==1.12 tenacity==8.3.0 threadpoolctl==3.5.0 tiktoken==0.6.0 tokenizers==0.19.1 torch==2.2.0+cpu tqdm==4.66.4 transformers==4.40.2 typing-inspect==0.9.0 typing_extensions==4.11.0 tzdata==2024.1 ujson==5.9.0 urllib3==2.2.1 voyager==2.0.6 Werkzeug==3.0.3 wrapt==1.16.0 xxhash==3.4.1 yarl==1.9.4
The code in question is : def create_or_get_colbert_model(username): index_name=username+"_report_history" index_path = PATH/.ragatouille/colbert/indexes/"+index_name+"/" print("Index path = ",index_path) if os.path.exists(index_path): RAG = RAGPretrainedModel.from_index(index_path) else: RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0") return RAG
After printing "Index path = " , index_path, I get : Loading segmented_maxsim_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
This happens even after creating a fresh ENV, and fresh installs of torch-cpu-2.2.0, sentence_transformers, transformers and ragatouille.