Do you guys have some notes on the settings for local llm configs?

Do we leave API key blank? Assuming we use the local Ollama URL in both configs?

This is running on an M3 Mac.

Here's what I have in mine:

`NICK_NAME = 'Brad' # This is your nick name. Make sure to set it at the beginning and don't change so that LLM will not get confused.

EMBEDDING_BASE_URL = 'http://127.0.0.1:11434/' EMBEDDING_API_KEY = '' EMBEDDING_MODEL_NAME = "mixtral"

CHAT_BASE_URL = 'http://127.0.0.1:11434/' # Modify to your OpenAI compatible API url CHAT_API_KEY = '' CHAT_MODEL_NAME = "mixtral" CHAT_MAX_TOKEN = 64000

Path to the local directory of your Markdown notebook to store context information

CHAT_PATH = '../Users/bradn/Documents/Logseq/elefiechats' NOTE_PATH = '../Users/bradn/Documents/Logseq'

If your MarkDown notebook is serving with HTTPS, setup this URL so that you can click into the notebook page attached on reference.

CHAT_URL = 'https://localhost:3000/chat_history/' NOTE_URL = 'https://localhost:3000/notes/'`

When I load the UI and put in a chat, here is what I get. It seems to be replying in terminal, but nothing except my chat shows in theweb UI.

(base) bradn@Brad-MB backend % python app.py bm25 corpus updated INFO: Started server process [19262] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://localhost:5000 (Press CTRL+C to quit) call from admin

USER: What was my last note about?

[{'role': 'system', 'content': 'You are Loyal Elephie, Brad\'s autonomous secretary who has access to the following tools:\n1. You have an inner monologue section which could help you analyze the problem without disturbing Brad. To use inner monologue section, write your monologue between tags "" and "". The monologue should including the user problem breakdown the questions you don\'t yet understand. This tool is how you comprehend.\n\n2. You have a memory including Brad\'s notes and your past conversations with him, which could possibly provide useful context for this interaction. To use this external memory, write search query strings each per line between tags "

" and "". Provide precise date into the query if possible. This tool is how your recall.\nExample of using the memory:\nUser: Should I buy a new computer?\n\nBrad computer problem\nBrad buy new computer preference\n\nIf you see the search result, be mindful that the context could be ranging from a long period and they will be shown in a timely order.\n\n3. Once you have thoroughly comprehended the latest user input, respond by placing your message between the tags?<REPLY>?and?</REPLY>. Only the text inside the "" block will be visible to Brad. Your reply should be supportive, with an analytical, creative, extroverted, and playful personality. You love jokes, sarcasm, and making wild guesses while staying truthful to the accessible context when not making guesses. Always address Brad as "you". This tool is how you speak.\n\n\nBelow your interactions with the user (Brad) begin. You will also receive occasional system messages with situational information and instructions.\nCurrent time is 2024-06-06 10:47:10\n'}, {'role': 'user', 'content': "I don't remember how Loyal Elephie was created."}, {'role': 'assistant', 'content': 'To assist Brad, I need to search my memory for the questions:\nHow Loyal Elephie was created?\nHow AI secretary like Loyal Elephie was developed? '}, {'role': 'assistant', 'content': '\nLoyal Elephie created detail\nBrad AI secretary develop'}, {'role': 'system', 'content': '---begin search result---\n\nBrad mentioned that the current AI secretary -- Loyal Elephie is integrated with advanced vector search and LLM technology. It could be used to provide insightful advices because the AI secretary has access to vast knowledge from Brad\'s notes and conversations.\n---end search result---'}, {'role': 'assistant', 'content': 'Hey Brad, how could you forget about my creation? I am your artwork using advanced vector search and LLM technology. If you need insightful advices based on your notes and conversations between us, just tap on me, ah-ha!'}, {'role': 'user', 'content': 'What was my last note about?'}] 2024-06-06 10:47:12 INFO: ::1:55044 - "POST /v1/chat/completions HTTP/1.1" 500 ERROR: Exception in ASGI application Traceback (most recent call last): File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions yield File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_transports/default.py", line 233, in handle_request resp = self._pool.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request raise exc from None File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request response = connection.handle_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 101, in handle_request return self._connection.handle_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 143, in handle_request raise exc File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 93, in handle_request self._send_request_headers(**kwargs) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 151, in _send_request_headers with map_exceptions({h11.LocalProtocolError: LocalProtocolError}): File "/Users/bradn/miniconda3/lib/python3.11/contextlib.py", line 155, in exit self.gen.throw(typ, value, traceback) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.LocalProtocolError: Illegal header value b'Bearer '

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 880, in _request response = self._client.send( ^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_client.py", line 914, in send response = self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_client.py", line 942, in _send_handling_auth response = self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_client.py", line 979, in _send_handling_redirects response = self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_client.py", line 1015, in _send_single_request response = transport.handle_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_transports/default.py", line 232, in handle_request with map_httpcore_exceptions(): File "/Users/bradn/miniconda3/lib/python3.11/contextlib.py", line 155, in exit self.gen.throw(typ, value, traceback) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.LocalProtocolError: Illegal header value b'Bearer '

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/bradn/miniconda3/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/applications.py", line 116, in call await self.middleware_stack(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in call raise exc File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in call await self.app(scope, receive, _send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in call await self.app(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app raise exc File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app await app(scope, receive, sender) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/routing.py", line 746, in call await route.handle(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/routing.py", line 75, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/_exception_handler.py", line 55, in wrapped_app raise exc File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/_exception_handler.py", line 44, in wrapped_app await app(scope, receive, sender) File "/Users/bradn/miniconda3/lib/python3.11/site-packages/starlette/routing.py", line 70, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/fastapi/routing.py", line 299, in app raise e File "/Users/bradn/miniconda3/lib/python3.11/site-packages/fastapi/routing.py", line 294, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(*values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/Documents/dev/elephie/Loyal-Elephie/backend/memory_server.py", line 278, in create_chat_completion completion_or_chunks = client.chat.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_utils/_utils.py", line 271, in wrapper return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 643, in create return self._post( ^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 1091, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 852, in request return self._request( ^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 899, in _request return self._retry_request( ^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 961, in _retry_request return self._request( ^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 899, in _request return self._retry_request( ^^^^^^^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 961, in _retry_request return self._request( ^^^^^^^^^^^^^^ File "/Users/bradn/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 908, in _request raise APIConnectionError(request=request) from err openai.APIConnectionError: Connection error.

Here's what I get on frontend terminal:

(base) bradn@Brad-MB frontend % npm run start

ai-chatbot-starter@0.1.0 start PORT=8080 next start

ready - started server on 0.0.0.0:8080, url: http://localhost:8080 (node:79833) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead. (Use node --trace-deprecation ... to show where the warning was created) [Error: OpenAI API returned an error]

v2rockets / Loyal-Elephie

Local LLM Configuration #8

Path to the local directory of your Markdown notebook to store context information

If your MarkDown notebook is serving with HTTPS, setup this URL so that you can click into the notebook page attached on reference.

USER: What was my last note about?