mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
https://localai.io
MIT License
23.6k stars 1.8k forks source link

make prepare-extra-conda-environments wont work for M series mac #1520

Open harvy4002 opened 9 months ago

harvy4002 commented 9 months ago

LocalAI version:

v2.3.1-1-gc1888a8 (c1888a8062fc7a000457714d8426584d42d0f142)

Environment, CPU architecture, OS, and Version:

Darwin 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:28:45 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6020 arm64 Basically M2 Mac

Describe the bug I'm trying to use this comment here to get embeddings working with huggingface on my M2 mac https://github.com/mudler/LocalAI/issues/780#issuecomment-1654599333 the new make file as of version 2+ seems to be locked to linux versions of dependencies so it doesn't compile. I have conda installed, etc. What exactly do I need for huggingface-embeddings ? happy for this to move to discussion if needed

To Reproduce

run make prepare-extra-conda-environments on M series Macs

Expected behavior

The environment to be prepared correctly for M1 macs but ultimately I just want to get huggingface-embeddings working

Logs

harvy@Harvinders-MBP LocalAI % make prepare-extra-conda-environments                                             
/Library/Developer/CommandLineTools/usr/bin/make -C backend/python/autogptq
Creating virtual environment...
Channels:
 - defaults
 - conda-forge
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - _libgcc_mutex
  - _openmp_mutex==5.1=1_gnu
  - bzip2==1.0.8=h7b6447c_0
  - ca-certificates==2023.08.22=h06a4308_0
  - ld_impl_linux-64==2.38=h1181459_1
  - libffi==3.4.4=h6a678d5_0
  - libgcc-ng==11.2.0=h1234567_1
  - libgomp==11.2.0=h1234567_1
  - libstdcxx-ng==11.2.0=h1234567_1
  - libuuid==1.41.5=h5eee18b_0
  - ncurses==6.4=h6a678d5_0
  - openssl==3.0.11=h7f8727e_2
  - pip==23.2.1=py311h06a4308_0
  - python==3.11.5=h955ad1f_0
  - readline==8.2=h5eee18b_0
  - setuptools==68.0.0=py311h06a4308_0
  - sqlite==3.41.2=h5eee18b_0
  - tk==8.6.12=h1ccaba5_0
  - wheel==0.41.2=py311h06a4308_0
  - xz==5.4.2=h5eee18b_0
  - zlib==1.2.13=h5eee18b_0

Current channels:

  - https://repo.anaconda.com/pkgs/main/osx-arm64
  - https://repo.anaconda.com/pkgs/r/osx-arm64
  - https://conda.anaconda.org/conda-forge/osx-arm64

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

make[1]: *** [autogptq] Error 1
make: *** [prepare-extra-conda-environments] Error 2

curl http://localhost:8080/v1/embeddings -H "Content-Type: application/json" -d '{
"model": "text-embedding-ada-002",
"input": "Test"}'

{"error":{"code":500,"message":"grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/huggingface-embeddings. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS","type":""}}%`

Additional context

vlbosch commented 6 months ago

@harvy4002 Did you find a solution? I am running into the exact same problem with the latest LocalAI on my M3 Mac. Thanks in advance!