search

cpacker / MemGPT

Letta (fka MemGPT) is a framework for creating stateful LLM services.
https://letta.com
Apache License 2.0
12.05k stars 1.33k forks source link

ADE Select Model - Doesnt show the list of openai models #1897

Open raolak opened 2 weeks ago

raolak commented 2 weeks ago

Describe the bug I have set environment variables while running docker compose in my local setup. But in the ADE UI I see only letta-free option under dropdown. My understanding if I set llm model for open ai then tha should get listed. Can you please help with steps to see all the model dropdowsn in the ADE UI while creating the agent?

Please describe your setup FOllowed this document to setup letta using docker compose https://docs.letta.com/install

Environment variables within docker container

env

LETTA_PG_DB=letta LETTA_LLM_MODEL=gpt-4 HOSTNAME=letta SERPAPI_API_KEY= PYTHON_PIP_VERSION=24.0 HOME=/root OLDPWD=/letta LETTA_ENVIRONMENT=PRODUCTION LETTA_LLM_ENDPOINT=https://api.openai.com/v1 LETTA_LLM_CONTEXT_WINDOW=None OPENAI_API_KEY= GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305 LETTA_EMBEDDING_ENDPOINT_TYPE=openai PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/dbf0c85f76fb6e1ab42aa672ffca6f0a675d9ee4/public/get-pip.py TERM=xterm PATH=/app/.venv/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LETTA_EMBEDDING_MODEL=text-embedding-ada-002 LANG=C.UTF-8 LETTA_PG_PASSWORD=letta LETTA_EMBEDDING_DIM=None PYTHON_VERSION=3.12.2 LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1 LETTA_PG_HOST=pgvector_db LETTA_PG_USER=letta VIRTUAL_ENV=/app/.venv PWD=/root/.letta PYTHON_GET_PIP_SHA256=dfe9fd5c28dc98b5ac17979a953ea550cec37ae1b47a5116007395bfacff2ab9 LETTA_LLM_ENDPOINT_TYPE=openai LETTA_PG_PORT=5432

Startup logs 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31061 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31066 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31065 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31064 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31063 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:04:54 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)]

Complete logs 2024-10-16 15:02:45 Starting MEMGPT server... 2024-10-16 15:02:50 uvicorn.error - INFO - Started server process [8] 2024-10-16 15:02:50 uvicorn.error - INFO - Waiting for application startup. 2024-10-16 15:02:50 uvicorn.error - INFO - Application startup complete. 2024-10-16 15:02:50 uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8083 (Press CTRL+C to quit) 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET / HTTP/1.1" 200 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET /index-9a9c449b.js HTTP/1.1" 200 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31050 - "GET /index-3ab03d5b.css HTTP/1.1" 200 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET /favicon.ico HTTP/1.1" 404 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31054 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31053 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31052 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:03:17 uvicorn.access - INFO - 192.168.65.1:31051 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:03:18 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:03:18 uvicorn.access - INFO - 192.168.65.1:31050 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:03:19 uvicorn.access - INFO - 192.168.65.1:31056 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 15:03:19 uvicorn.access - INFO - 192.168.65.1:31055 - "GET /v1/jobs/active HTTP/1.1" 200 2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31055 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31057 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31059 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31058 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:03:20 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:03:20 uvicorn.access - INFO - 192.168.65.1:31056 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:03:21 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:03:21 uvicorn.access - INFO - 192.168.65.1:31056 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:04:53 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31061 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31066 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31065 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31064 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:04:53 uvicorn.access - INFO - 192.168.65.1:31063 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:04:54 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:04:54 uvicorn.access - INFO - 192.168.65.1:31062 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:27:51 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:27:51 uvicorn.access - INFO - 192.168.65.1:31096 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31095 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31097 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31098 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:27:52 uvicorn.access - INFO - 192.168.65.1:31093 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:27:53 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:27:53 uvicorn.access - INFO - 192.168.65.1:31094 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:27:59 uvicorn.access - INFO - 192.168.65.1:31100 - "GET /v1/jobs/active HTTP/1.1" 200 2024-10-16 15:27:59 uvicorn.access - INFO - 192.168.65.1:31099 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31101 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31103 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31099 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:28:00 uvicorn.access - INFO - 192.168.65.1:31100 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:28:01 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:28:01 uvicorn.access - INFO - 192.168.65.1:31100 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:28:01 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:28:01 uvicorn.access - INFO - 192.168.65.1:31102 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:33:02 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31121 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31119 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31120 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31116 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31117 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:33:02 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:33:02 uvicorn.access - INFO - 192.168.65.1:31118 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:33:04 uvicorn.access - INFO - 192.168.65.1:31123 - "GET /v1/jobs/active HTTP/1.1" 200 2024-10-16 15:33:04 uvicorn.access - INFO - 192.168.65.1:31122 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 15:33:23 uvicorn.access - INFO - 192.168.65.1:31124 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:33:25 uvicorn.access - INFO - 192.168.65.1:31125 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:33:46 uvicorn.access - INFO - 192.168.65.1:31126 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:33:47 uvicorn.access - INFO - 192.168.65.1:31126 - "GET /v1/jobs/active HTTP/1.1" 200 2024-10-16 15:33:47 uvicorn.access - INFO - 192.168.65.1:31127 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31128 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31131 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31132 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:33:49 uvicorn.access - INFO - 192.168.65.1:31129 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:33:50 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:33:50 uvicorn.access - INFO - 192.168.65.1:31130 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:33:51 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:33:51 uvicorn.access - INFO - 192.168.65.1:31130 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:34:12 uvicorn.access - INFO - 192.168.65.1:31133 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 15:34:12 uvicorn.access - INFO - 192.168.65.1:31134 - "GET /v1/jobs/active HTTP/1.1" 200 2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31133 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31137 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31136 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:34:13 uvicorn.access - INFO - 192.168.65.1:31134 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:34:14 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:34:14 uvicorn.access - INFO - 192.168.65.1:31135 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:34:15 uvicorn.access - INFO - 192.168.65.1:31135 - "GET /v1/jobs/active HTTP/1.1" 200 2024-10-16 15:34:15 uvicorn.access - INFO - 192.168.65.1:31138 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31135 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31139 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31140 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:34:16 uvicorn.access - INFO - 192.168.65.1:31138 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:34:17 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:34:17 uvicorn.access - INFO - 192.168.65.1:31141 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31144 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31143 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31145 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:34:48 uvicorn.access - INFO - 192.168.65.1:31142 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:34:49 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:34:49 uvicorn.access - INFO - 192.168.65.1:31146 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31151 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31148 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31147 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:41:24 uvicorn.access - INFO - 192.168.65.1:31150 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 15:41:25 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:41:25 uvicorn.access - INFO - 192.168.65.1:31149 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:41:26 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:41:26 uvicorn.access - INFO - 192.168.65.1:31149 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:41:37 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:41:37 uvicorn.access - INFO - 192.168.65.1:31152 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:42:29 Added record with id message-5f5e919f-9b8c-4947-9551-a690095f4bb1 2024-10-16 15:42:29 Added record with id message-34ec5961-5eb7-419d-9e84-f0b278e637e1 2024-10-16 15:42:29 Added record with id message-1a6d60b3-d6da-4029-aa8e-38a3860162d2 2024-10-16 15:42:29 Added record with id message-234f4672-f380-4d20-b5bb-9938493afed2 2024-10-16 15:42:29 Added record with id message-3cc56703-6901-4af7-8e27-7949b0ef5814 2024-10-16 15:42:29 uvicorn.access - INFO - 192.168.65.1:31153 - "POST /v1/agents/ HTTP/1.1" 200 2024-10-16 15:42:29 uvicorn.access - INFO - 192.168.65.1:31153 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 15:43:33 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31154 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31156 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200 2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31159 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31158 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages?limit=1000&msg_object=true HTTP/1.1" 200 2024-10-16 15:43:33 Letta.letta.server.server - ERROR - Agent (user=user-00000000, agent=agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96) is already loaded 2024-10-16 15:43:33 NoneType: None 2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31157 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96 HTTP/1.1" 200 2024-10-16 15:43:33 Letta.letta.server.server - ERROR - Agent (user=user-00000000, agent=agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96) is already loaded 2024-10-16 15:43:33 NoneType: None 2024-10-16 15:43:33 uvicorn.access - INFO - 192.168.65.1:31154 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/archival HTTP/1.1" 200 2024-10-16 15:43:34 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 15:43:34 uvicorn.access - INFO - 192.168.65.1:31155 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 15:43:40 Warning: token streaming is only supported for OpenAI models. Setting to False. 2024-10-16 15:43:40 uvicorn.access - INFO - 192.168.65.1:31160 - "POST /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages HTTP/1.1" 200 2024-10-16 15:43:40 Added record with id message-2041d1ce-6231-4504-8431-06eb170871bc 2024-10-16 15:43:45 Added record with id message-297022c9-d6be-4b4e-ab18-1c17ab46b314 2024-10-16 15:43:45 Added record with id message-60080407-5148-4578-b18d-5afaa85b45fc 2024-10-16 15:43:45 Added record with id message-790a4dfe-eb4d-4f0b-8d11-141d13b3b97b 2024-10-16 15:44:04 uvicorn.access - INFO - 192.168.65.1:31161 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200 2024-10-16 15:44:04 uvicorn.access - INFO - 192.168.65.1:31162 - "GET /v1/sources/ HTTP/1.1" 200 2024-10-16 16:00:44 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31186 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31186 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200 2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31185 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31184 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96 HTTP/1.1" 200 2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31183 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/archival HTTP/1.1" 200 2024-10-16 16:00:44 uvicorn.access - INFO - 192.168.65.1:31187 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages?limit=1000&msg_object=true HTTP/1.1" 200 2024-10-16 16:00:45 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 16:00:45 uvicorn.access - INFO - 192.168.65.1:31182 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 16:01:40 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31200 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31200 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/sources HTTP/1.1" 200 2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31201 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31205 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/archival HTTP/1.1" 200 2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31204 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96 HTTP/1.1" 200 2024-10-16 16:01:40 uvicorn.access - INFO - 192.168.65.1:31203 - "GET /v1/agents/agent-8d0189d7-f8ba-4fa6-8714-bab73d370b96/messages?limit=1000&msg_object=true HTTP/1.1" 200 2024-10-16 16:01:41 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 16:01:41 uvicorn.access - INFO - 192.168.65.1:31202 - "GET /v1/models/ HTTP/1.1" 200 2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31210 - "GET /v1/blocks/?label=persona HTTP/1.1" 200 2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31206 - "GET /v1/tools/ HTTP/1.1" 200 2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31207 - "GET /v1/blocks/?label=human HTTP/1.1" 200 2024-10-16 16:01:45 uvicorn.access - INFO - 192.168.65.1:31208 - "GET /v1/agents/ HTTP/1.1" 200 2024-10-16 16:01:46 [EmbeddingConfig(embedding_endpoint_type='hugging-face', embedding_endpoint='https://embeddings.memgpt.ai', embedding_model='letta-free', embedding_dim=1024, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None), EmbeddingConfig(embedding_endpoint_type='openai', embedding_endpoint='https://api.openai.com/v1', embedding_model='text-embedding-ada-002', embedding_dim=1536, embedding_chunk_size=300, azure_endpoint=None, azure_version=None, azure_deployment=None)] 2024-10-16 16:01:46 uvicorn.access - INFO - 192.168.65.1:31208 - "GET /v1/models/embedding HTTP/1.1" 200 2024-10-16 16:01:46 [LLMConfig(model='letta-free', model_endpoint_type='openai', model_endpoint='https://inference.memgpt.ai', model_wrapper=None, context_window=16384)] 2024-10-16 16:01:46 uvicorn.access - INFO - 192.168.65.1:31209 - "GET /v1/models/ HTTP/1.1" 200

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

Letta Config Please attach your ~/.letta/config file or copy past it below.

Screenshot 2024-10-16 at 4 07 28 PM

Appreciate if any one can help here...this is blocking demo :(

sarahwooders commented 2 weeks ago

Can you make sure you have the latest docker image? You can run docker pull lettaai/letta:latest. I just tried with OpenAI and wasn't able to reproduce the issue, so this might be a problem with an older version?

Also for your enviornment variables, you should also need to set OPENAI_API_KEY. The following variables are deprecated so won't be read/used: 

LETTA_LLM_MODEL=gpt-4
LETTA_LLM_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_CONTEXT_WINDOW=None
LETTA_EMBEDDING_ENDPOINT_TYPE=openai

LETTA_EMBEDDING_MODEL=text-embedding-ada-002
LETTA_EMBEDDING_DIM=None
LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_ENDPOINT_TYPE=openai

Also if you have an older version of the DB initialized, you may need to run rm -r ./.persist/pgdata.

raolak commented 2 weeks ago

Hi Saraha,

Thank you for the quick response—I really appreciate it!

My goal is to deploy the Letta server on Kubernetes. Below are some challenges I encountered during the process. I managed to get the server running on Kubernetes earlier (before the rebranding), but I’m now reinstalling it with the latest version. Here's what I’ve done so far:

  1. I initially tried using the lettaai/letta:latest image, but it seems incompatible with my Mac microk8s architecture. Could you confirm if this image supports all architectures? If needed, I’m happy to help create a compatible version.
  2. I built the image from the repo’s Dockerfile and pushed it to my local registry.
  3. My Helm chart currently copies config and secret files to /root/.letta. I’m unsure if this step is still necessary with the latest version.
  4. I pass all necessary environment variables, including DB-specific ones, through the deployment YAML. I suspect I missed setting the database name, which is causing Letta to default to a local SQLite DB—I'm investigating this.

Here’s the environment configuration I’m using inside the container:

LETTA_LLM_MODEL=gpt-4o LETTA_ENVIRONMENT=PRODUCTION LETTA_LLM_CONTEXT_WINDOW=8192 OPENAI_API_KEY=sk-xxx LETTA_LLM_ENDPOINT=https://api.openai.com/v1 LETTA_EMBEDDING_CHUNK_SIZE=300 LETTA_EMBEDDING_ENDPOINT_TYPE=openai LETTA_DEBUG=true LETTA_EMBEDDING_MODEL=text-embedding-ada-002 LETTA_PG_HOST=postgres-db (k8s service name) LETTA_PG_PASSWORD=xxx LETTA_PG_USER=letta LETTA_PG_PORT=5432 PYTHON_VERSION=3.12.2 LETTA_EMBEDDING_DIM=1536 LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1 LETTA_SERVER_PASS=xxx LETTA_LLM_ENDPOINT_TYPE=openai

Clarifications

  1. Do I need to deploy a separate server for each model, with corresponding environment variables for each?
  2. Is there a way to configure models without relying on environment variables?
  3. I’m exploring the use of the data source feature to generate boilerplate schemas. Is this feasible? We can dive deeper into this after I sort out other issues.

I’d love to have a quick call to discuss my platform use cases. Letta is a critical enabler for my platform, and I appreciate all the great work your team is doing🙏. I’m also happy to contribute to any efforts that help with improvements or fixes.

Thanks again!

Best, Inr

On Thu, Oct 17, 2024 at 12:45 AM Sarah Wooders @.***> wrote:

Can you make sure you have the latest docker image? You can run docker pull lettaai/letta:latest. I just tried with OpenAI and wasn't able to reproduce the issue, so this might be a problem with an older version?

Also for your enviornment variables, you should also need to set OPENAI_API_KEY. The following variables are deprecated so won't be read/used:

LETTA_LLM_MODEL=gpt-4 LETTA_LLM_ENDPOINT=https://api.openai.com/v1 LETTA_LLM_CONTEXT_WINDOW=None LETTA_EMBEDDING_ENDPOINT_TYPE=openai https://api.openai.com/v1LETTA_LLM_CONTEXT_WINDOW=NoneLETTA_EMBEDDING_ENDPOINT_TYPE=openai

LETTA_EMBEDDING_MODEL=text-embedding-ada-002 LETTA_EMBEDDING_DIM=None LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1 LETTA_LLM_ENDPOINT_TYPE=openai https://api.openai.com/v1LETTA_LLM_ENDPOINT_TYPE=openai

Also if you have an older version of the DB initialized, you may need to run rm -r ./.persist/pgdata.

— Reply to this email directly, view it on GitHub https://github.com/cpacker/MemGPT/issues/1897#issuecomment-2417740788, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGPZOKK3AWXSEVVGWGZJKLZ323MZAVCNFSM6AAAAABQBE27WGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXG42DANZYHA . You are receiving this because you authored the thread.Message ID: @.***>

raolak commented 2 weeks ago

Can you make sure you have the latest docker image? You can run docker pull lettaai/letta:latest. I just tried with OpenAI and wasn't able to reproduce the issue, so this might be a problem with an older version?

Also for your enviornment variables, you should also need to set OPENAI_API_KEY. The following variables are deprecated so won't be read/used:

LETTA_LLM_MODEL=gpt-4
LETTA_LLM_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_CONTEXT_WINDOW=None
LETTA_EMBEDDING_ENDPOINT_TYPE=openai

LETTA_EMBEDDING_MODEL=text-embedding-ada-002
LETTA_EMBEDDING_DIM=None
LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1
LETTA_LLM_ENDPOINT_TYPE=openai

Also if you have an older version of the DB initialized, you may need to run rm -r ./.persist/pgdata.

I will re install again today using these steps and will keep you posted. Thank you !

raolak commented 2 weeks ago

I re installed with setting right environment variables...I was able to deploy letta in k8s... As mentioned, I have built the docker image and not used lettaai/letta:latest from docker hub as it had architecture compatibility issue

But I dont see gpt-4 model listing in the ADE create agent UI.

Here is :LETTA and OPEN AI specific the env from k8s pod

LETTA_EMBEDDING_MODEL=text-embedding-ada-002 LETTA_EMBEDDING_ENDPOINT_TYPE=openai LETTA_EMBEDDING_CHUNK_SIZE=300 LETTA_EMBEDDING_ENDPOINT=https://api.openai.com/v1 LETTA_EMBEDDING_DIM=None

LETTA_LLM_MODEL=gpt-4 LETTA_LLM_CONTEXT_WINDOW=None LETTA_LLM_ENDPOINT_TYPE=openai LETTA_LLM_ENDPOINT=https://api.openai.com/v1

LETTA_ENVIRONMENT=PRODUCTION LETTA_DEBUG=true

OPENAI_API_KEY=

LETTA_PG_DB=jnaa-ai LETTA_PG_USER=jnaa LETTA_PG_HOST=jnaa-postgres-jnaa-postgres-db LETTA_PG_PORT=5432 LETTA_PG_PASSWORD=WsxQaz21!

POSTGRES_URI=postgresql+pg8000://jnaa:WsxQaz21!@jnaa-postgres-jnaa-postgres-db:5432/jnaa-ai

Also /root/.letta. configs and credentials

/root/.letta/config [defaults] preset = memgpt_chat persona = sam_pov human = basic

[archival_storage] type = postgres path = /root/.letta/chroma uri = postgresql+pg8000://jnaa:XXXX!@jnaa-postgres-jnaa-postgres-db:5432/jnaa-ai

[recall_storage] type = postgres path = /root/.letta uri = postgresql+pg8000://jnaa:XXX!@jnaa-postgres-jnaa-postgres-db:5432/jnaa-ai

[metadata_storage] type = sqlite path = /root/.letta

[version] letta_version = 0.4.1

[client] anon_clientid = user-00000000

/root/.letta/credentials

[openai] auth_type = bearer_token key = sk-....

[azure] auth_type = api_key

Clarification

raolak commented 1 week ago

I was unable to get this working for gpt-4. I am re installing again tomorrow and keep the progress update here !