Open rkarteria opened 2 months ago
JSv4
commented
2 months ago I will take a look at this. I think it's something mac-related because I've also seen it reported in a couple of other projects for people running Apple silicon. I don't have a Mac tho so may ask for your help testing a fix if ur game.
rkarteria
commented
2 months ago Absolutely, would be more than happy to help!
rkarteria
commented
2 months ago Sorry a bit of a dumb follow up, I was able to get around this by removing all the references to:
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
So I went through the now running docker container and I tried finding libgomp-4dbbc2f2.so.1.0.0. But didn't have any luck as /usr/local/lib/python3.10/site-packages/torch/lib/ only contained (Sorry im not really familiar with how these packages are compiled):
Sorry im really unfamiliar with this, but it looks like this is more of a torch / hugging face issue!
JSv4
commented
2 months ago Yeah, @rkarteria, I think it's a combination of mac and hugging face. Not sure why it's an issue inside the docker container though... Sorry, I'm not that familiar with OSX environment and haven't come across this issue. If you have the time, I'd try to keep poking around issues lists on HuggingFace or LLamaIndex repos,
rjrobben
commented
1 month ago I am able get around on a m3 machine by adding
environment:
- LD_PRELOAD=/usr/local/lib/python3.10/site-packages/torch/lib/../../torch.libs/libgomp-4dbbc2f2.so.1.0.0to the local.yml:
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django: &django
build:
context: .
args:
- GITHUB_ACTIONS
dockerfile: ./compose/local/django/Dockerfile
image: opencontractserver_local_django
depends_on:
- postgres
- redis
- vector-embedder
- nlm-ingestor
volumes:
- .:/app:z
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
deploy:
resources:
limits:
memory: 18G
environment:
- LD_PRELOAD=/usr/local/lib/python3.10/site-packages/torch/lib/../../torch.libs/libgomp-4dbbc2f2.so.1.0.0
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
container_name: postgres
image: opencontractserver_production_postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data:Z
- local_postgres_data_backups:/backups:z
env_file:
- ./.envs/.local/.postgres
redis:
image: redis:6
container_name: redis
nlm-ingestor:
image: jscrudato/nlm-ingestor-opencontracts
container_name: nlm-ingestor
vector-embedder:
image: jscrudato/vector-embedder-microservice
container_name: vector-embedder
environment:
PORT: 8000
celeryworker:
<<: *django
image: opencontractserver_local_celeryworker
depends_on:
- redis
- postgres
ports: []
command: /start-celeryworker
celerybeat:
<<: *django
image: opencontractserver_local_celerybeat
container_name: celerybeat
depends_on:
- redis
- postgres
ports: []
command: /start-celerybeat
flower:
<<: *django
image: opencontractserver_local_flower
container_name: flower
ports:
- "5555:5555"
command: /start-flower
frontend:
build:
context: ./frontend
dockerfile: ./Dockerfile
image: opencontractserver_local_frontend
container_name: frontend
depends_on:
- django
env_file:
- ./.envs/.local/.frontend
ports:
- "3000:3000"
profiles:
- "fullstack"Really not sure why different os will affect sth happening inside docker. Only can think of different file encoding.
JSv4
commented
1 month ago Thanks @rjrobben. Yeah that's super weird, right? We'll thanks for finding a fix. I'm not going to make any changes to main until I understand why this is happening, but I don't have an OSX dev env. Will leave this open in the hopes others can shed more light on this.
Hello!
While running the quickstart I happened to get this error, which exists within
opencontracts-django-1,flower,celerybeat. (I am running this on Sonoma 14.1 using an M3.) The full trace is:I saw that this error is relatively common [1], [2] and [3] and I think it should be fixed using:
LD_PRELOAD=<...>Although I am not totally sure and would love any feedback!