Open vishalmish opened 1 year ago
Have you tried to edit the EMBED_DIM in .env config to 5120? Here:
EMBED_DIM = 8192
The ggml-vicuna-13b-4bit.bin model uses this embed dimemsion.
ok, so we miss the com.docker.compose.project.environment_file
file label, which should have been set to point to your .env
file, but is only set when an explicit --enf-file
flag is passed. We need to fix this, or ensure Docker Desktop UI don't need this. This typically will be possible once we implement https://github.com/docker/compose-cli/issues/984
Originally posted by @ndeloof in https://github.com/docker/for-mac/issues/5745#issuecomment-860638751
Please remove spaces from the .env settings, that is invalid in .bash
Should look like this instead
EMBED_DIM=8192
Duplicates
Steps to reproduce 🕹
ggml-vicuna-13b-4bit.bin
model.env
file (From the default)SMART_LLM_MODEL=./models/ggml-vicuna-13b-4bit.bin FAST_LLM_MODEL=./models/ggml-vicuna-13b-4bit.bin EMBED_DIM = 8192
docker run -p80:3000 foo/auto-llama Traceback (most recent call last): File "/app/main.py", line 3, in
import commands as cmd
File "/app/commands.py", line 1, in
import browse
File "/app/browse.py", line 4, in
from llm_utils import create_chat_completion
File "/app/llm_utils.py", line 4, in
cfg = Config()
^^^^^^^^
File "/app/config.py", line 18, in call
cls._instances[cls] = super(
^^^^^^
File "/app/config.py", line 69, in init
self.EMBED_DIM = int(os.getenv("EMBED_DIM"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
(base)