Couldn't load model

[annidy]

Traceback (most recent call last):
  File "/Users/xxx/Project/voice-assistant/main.py", line 155, in <module>
    llm = LlamaCpp(
          ^^^^^^^^^
  File "/Users/xxx/miniconda/envs/VoiceAI/lib/python3.11/site-packages/langchain_core/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "/Users/xxx/miniconda/envs/VoiceAI/lib/python3.11/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for LlamaCpp
__root__
  Could not load Llama model from path: /Users/xxx/Project/voice-assistant/yi-chat-6B-GGUF/yi-chat-6b.Q8_0.gguf. Received error  (type=value_error)

System envrioment:

uname -a
Darwin ady.local 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul  5 22:21:56 PDT 2023; root:xnu-8796.141.3~6/RELEASE_X86_64 x86_64

[zsstssjxzzyxgs]

Hi, I have met the same problem. Perhaps there is something wrong with our models. Here is my system envrioment:

uname -a
Darwin zhangzhexindeMacBook-Pro.local 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:34 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T8103 arm64

[annidy]

Hi, I have met the same problem. Perhaps there is something wrong with our models. Here is my system envrioment:

uname -a
Darwin zhangzhexindeMacBook-Pro.local 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:34 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T8103 arm64

I download this model from huggingface, and the checksum is correct. I think maybe it not support macbook pro, the author using mac studio

[ioctlsg]

It seem like you all are trying to run off x86. but this is for ARM if i am correct

[Yeeler]

I am running it on a MX Linux machine with 64gb memory. It can load the llm model, but failed on record_audio() method.

(VoiceAI) mx@nebula:~/_ai/voice-assistant $ uname -a Linux nebula 6.7.11-1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 6.7-17~mx23ahs (2024-03-27) x86_64 GNU/Linux (VoiceAI) mx@nebula:~/_ai/voice-assistant $ lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm (VoiceAI) mx@nebula:~/_ai/voice-assistant $ python main.py Listening... ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe ALSA lib pcm.c:2666:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side ALSA lib pcm_route.c:877:(find_matching_chmap) Found no matching channel map Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp ALSA lib pcm_oss.c:397:(_snd_pcm_oss_open) Cannot open device /dev/dsp ALSA lib dlmisc.c:337:(snd_dlobj_cache_get0) Cannot open shared library libasound_module_pcm_a52.so (/home/leo/miniconda3/envs/VoiceAI/lib/libstdc++.so.6: version GLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libjxl_threads.so.0.7)) ALSA lib dlmisc.c:337:(snd_dlobj_cache_get0) Cannot open shared library libasound_module_pcm_a52.so (/home/leo/miniconda3/envs/VoiceAI/lib/libstdc++.so.6: versionGLIBCXX_3.4.30' not found (required by /lib/x86_64-linux-gnu/libjxl_threads.so.0.7)) ALSA lib confmisc.c:160:(snd_config_get_card) Invalid field card ALSA lib pcm_usb_stream.c:482:(_snd_pcm_usb_stream_open) Invalid card 'card' ALSA lib confmisc.c:160:(snd_config_get_card) Invalid field card ALSA lib pcm_usb_stream.c:482:(_snd_pcm_usb_stream_open) Invalid card 'card' Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock Traceback (most recent call last): File "/home/leo/_ai/voice-assistant/main.py", line 152, in record_audio() File "/home/leo/_ai/voice-assistant/main.py", line 51, in record_audio data = stream.read(CHUNK) ^^^^^^^^^^^^^^^^^^ File "/home/leo/miniconda3/envs/VoiceAI/lib/python3.11/site-packages/pyaudio/init.py", line 570, in read return pa.read_stream(self._stream, num_frames, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [Errno -9981] Input overflowed