khoj-ai / khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (e.g gpt, claude, gemini, llama, qwen, mistral).
https://khoj.dev
GNU Affero General Public License v3.0
14.28k stars 708 forks source link

[Bug] The website can't working well on the Azure VM (Command Version) #454

Closed Gangxin-Li closed 1 year ago

Gangxin-Li commented 1 year ago

Hi Ai leaders,

I am using the Azure VM to build a service via command line, but there is a problem with the processor

Here is the result of the "khoj":

INFO 🌘 Starting Khoj main.py:66 WARNING 🚨 No Processor configuration available. configure.py:280 INFO 🔍 📜 Setting up text search model configure.py:146 INFO 🔍 🌄 Setting up image search model configure.py:150 INFO 💎 Setting up search for markdown notes configure.py:189

Here is the result of the "khoj --gui"

This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

I have no idea how to deploy it on the command line. Do you think it's problem? or just normal things when you use the command line to build the website.

Many thanks, Gangxin

sabaimran commented 1 year ago

Hey @Gangxin-Li , can you please try running it like khoj (without the --gui flag). This should still allow you to access the web pages, but it won't bother with the GUI packages.

Gangxin-Li commented 1 year ago

Hi @sabaimran, yes, I can run the server without using the --gui option. However, the same problem persists: I can't access the chat portal. It lacked the processor folder, and I add it manually, but it's still not working.

INFO 📬 Search models, Content index updated via API api.py:600 INFO 127.0.0.1:32778 - "GET /api/update?&client=web&force=true HTTP/1.1" 200 h11_impl.py:431 [13:23:08] INFO 127.0.0.1:33430 - "GET /chat HTTP/1.1" 200 h11_impl.py:431 INFO 127.0.0.1:33430 - "GET /api/chat/history?client=web HTTP/1.1" 500 h11_impl.py:431 ERROR Exception in ASGI application h11_impl.py:369

                ╭─────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────╮                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py:366 in run_asgi           │                
                │                                                                                                                         │                
                │   363 │   # ASGI exception wrapper                                                                                      │                
                │   364 │   async def run_asgi(self, app):                                                                                │                
                │   365 │   │   try:                                                                                                      │                
                │ ❱ 366 │   │   │   result = await app(self.scope, self.receive, self.send)                                               │                
                │   367 │   │   except BaseException as exc:                                                                              │                
                │   368 │   │   │   msg = "Exception in ASGI application\n"                                                               │                
                │   369 │   │   │   self.logger.error(msg, exc_info=exc)                                                                  │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py:75 in __call__           │                
                │                                                                                                                         │                
                │   72 │   │   │   │   │   port = 0                                                                                       │                
                │   73 │   │   │   │   │   scope["client"] = (host, port)  # type: ignore[index]                                          │                
                │   74 │   │                                                                                                              │                
                │ ❱ 75 │   │   return await self.app(scope, receive, send)                                                                │                
                │   76                                                                                                                    │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/fastapi/applications.py:269 in __call__                      │                
                │                                                                                                                         │                
                │   266 │   async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:                                 │                
                │   267 │   │   if self.root_path:                                                                                        │                
                │   268 │   │   │   scope["root_path"] = self.root_path                                                                   │                
                │ ❱ 269 │   │   await super().__call__(scope, receive, send)                                                              │                
                │   270 │                                                                                                                 │                
                │   271 │   def add_api_route(                                                                                            │                
                │   272 │   │   self,                                                                                                     │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/applications.py:124 in __call__                    │                
                │                                                                                                                         │                
                │   121 │                                                                                                                 │                
                │   122 │   async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:                                 │                
                │   123 │   │   scope["app"] = self                                                                                       │                
                │ ❱ 124 │   │   await self.middleware_stack(scope, receive, send)                                                         │                
                │   125 │                                                                                                                 │                
                │   126 │   # The following usages are now discouraged in favour of configuration                                         │                
                │   127 │   # during Starlette.__init__(...)                                                                              │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/middleware/errors.py:184 in __call__               │                
                │                                                                                                                         │                
                │   181 │   │   │   # We always continue to raise the exception.                                                          │                
                │   182 │   │   │   # This allows servers to log the error, or allows test clients                                        │                
                │   183 │   │   │   # to optionally raise the error within the test case.                                                 │                
                │ ❱ 184 │   │   │   raise exc                                                                                             │                
                │   185 │                                                                                                                 │                
                │   186 │   def format_line(                                                                                              │                
                │   187 │   │   self, index: int, line: str, frame_lineno: int, frame_index: int                                          │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/middleware/errors.py:162 in __call__               │                
                │                                                                                                                         │                
                │   159 │   │   │   await send(message)                                                                                   │                
                │   160 │   │                                                                                                             │                
                │   161 │   │   try:                                                                                                      │                
                │ ❱ 162 │   │   │   await self.app(scope, receive, _send)                                                                 │                
                │   163 │   │   except Exception as exc:                                                                                  │                
                │   164 │   │   │   request = Request(scope)                                                                              │                
                │   165 │   │   │   if self.debug:                                                                                        │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/exceptions.py:93 in __call__                       │                
                │                                                                                                                         │                
                │    90 │   │   │   │   handler = self._lookup_exception_handler(exc)                                                     │                
                │    91 │   │   │                                                                                                         │                
                │    92 │   │   │   if handler is None:                                                                                   │                
                │ ❱  93 │   │   │   │   raise exc                                                                                         │                
                │    94 │   │   │                                                                                                         │                
                │    95 │   │   │   if response_started:                                                                                  │                
                │    96 │   │   │   │   msg = "Caught handled exception, but response already started."                                   │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/exceptions.py:82 in __call__                       │                
                │                                                                                                                         │                
                │    79 │   │   │   await send(message)                                                                                   │                
                │    80 │   │                                                                                                             │                
                │    81 │   │   try:                                                                                                      │                
                │ ❱  82 │   │   │   await self.app(scope, receive, sender)                                                                │                
                │    83 │   │   except Exception as exc:                                                                                  │                
                │    84 │   │   │   handler = None                                                                                        │                
                │    85                                                                                                                   │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py:21 in __call__          │                
                │                                                                                                                         │                
                │   18 │   │   │   │   │   await self.app(scope, receive, send)                                                           │                
                │   19 │   │   │   │   except Exception as e:                                                                             │                
                │   20 │   │   │   │   │   dependency_exception = e                                                                       │                
                │ ❱ 21 │   │   │   │   │   raise e                                                                                        │                
                │   22 │   │   │   if dependency_exception:                                                                               │                
                │   23 │   │   │   │   # This exception was possibly handled by the dependency but it should                              │                
                │   24 │   │   │   │   # still bubble up so that the ServerErrorMiddleware can return a 500                               │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py:18 in __call__          │                
                │                                                                                                                         │                
                │   15 │   │   │   async with AsyncExitStack() as stack:                                                                  │                
                │   16 │   │   │   │   scope[self.context_name] = stack                                                                   │                
                │   17 │   │   │   │   try:                                                                                               │                
                │ ❱ 18 │   │   │   │   │   await self.app(scope, receive, send)                                                           │                
                │   19 │   │   │   │   except Exception as e:                                                                             │                
                │   20 │   │   │   │   │   dependency_exception = e                                                                       │                
                │   21 │   │   │   │   │   raise e                                                                                        │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/routing.py:670 in __call__                         │                
                │                                                                                                                         │                
                │   667 │   │   │   match, child_scope = route.matches(scope)                                                             │                
                │   668 │   │   │   if match == Match.FULL:                                                                               │                
                │   669 │   │   │   │   scope.update(child_scope)                                                                         │                
                │ ❱ 670 │   │   │   │   await route.handle(scope, receive, send)                                                          │                
                │   671 │   │   │   │   return                                                                                            │                
                │   672 │   │   │   elif match == Match.PARTIAL and partial is None:                                                      │                
                │   673 │   │   │   │   partial = route                                                                                   │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/routing.py:266 in handle                           │                
                │                                                                                                                         │                
                │   263 │   │   │   │   )                                                                                                 │                
                │   264 │   │   │   await response(scope, receive, send)                                                                  │                
                │   265 │   │   else:                                                                                                     │                
                │ ❱ 266 │   │   │   await self.app(scope, receive, send)                                                                  │                
                │   267 │                                                                                                                 │                
                │   268 │   def __eq__(self, other: typing.Any) -> bool:                                                                  │                
                │   269 │   │   return (                                                                                                  │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/routing.py:65 in app                               │                
                │                                                                                                                         │                
                │    62 │   async def app(scope: Scope, receive: Receive, send: Send) -> None:                                            │                
                │    63 │   │   request = Request(scope, receive=receive, send=send)                                                      │                
                │    64 │   │   if is_coroutine:                                                                                          │                
                │ ❱  65 │   │   │   response = await func(request)                                                                        │                
                │    66 │   │   else:                                                                                                     │                
                │    67 │   │   │   response = await run_in_threadpool(func, request)                                                     │                
                │    68 │   │   await response(scope, receive, send)                                                                      │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/fastapi/routing.py:227 in app                                │                
                │                                                                                                                         │                
                │    224 │   │   if errors:                                                                                               │                
                │    225 │   │   │   raise RequestValidationError(errors, body=body)                                                      │                
                │    226 │   │   else:                                                                                                    │                
                │ ❱  227 │   │   │   raw_response = await run_endpoint_function(                                                          │                
                │    228 │   │   │   │   dependant=dependant, values=values, is_coroutine=is_coroutine                                    │                
                │    229 │   │   │   )                                                                                                    │                
                │    230                                                                                                                  │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/fastapi/routing.py:162 in run_endpoint_function              │                
                │                                                                                                                         │                
                │    159 │   if is_coroutine:                                                                                             │                
                │    160 │   │   return await dependant.call(**values)                                                                    │                
                │    161 │   else:                                                                                                        │                
                │ ❱  162 │   │   return await run_in_threadpool(dependant.call, **values)                                                 │                
                │    163                                                                                                                  │                
                │    164                                                                                                                  │                
                │    165 def get_request_handler(                                                                                         │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/starlette/concurrency.py:41 in run_in_threadpool             │                
                │                                                                                                                         │                
                │   38 │   if kwargs:  # pragma: no cover                                                                                 │                
                │   39 │   │   # run_sync doesn't accept 'kwargs', so bind them in here                                                   │                
                │   40 │   │   func = functools.partial(func, **kwargs)                                                                   │                
                │ ❱ 41 │   return await anyio.to_thread.run_sync(func, *args)                                                             │                
                │   42                                                                                                                    │                
                │   43                                                                                                                    │                
                │   44 class _StopIteration(Exception):                                                                                   │                
                │                                                                                                                         │                
                │ /home/data/.local/lib/python3.8/site-packages/anyio/to_thread.py:33 in run_sync                                         │                
                │                                                                                                                         │                
                │   30 │   :return: an awaitable that yields the return value of the function.                                            │                
                │   31 │                                                                                                                  │                
                │   32 │   """                                                                                                            │                
                │ ❱ 33 │   return await get_asynclib().run_sync_in_worker_thread(                                                         │                
                │   34 │   │   func, *args, cancellable=cancellable, limiter=limiter                                                      │                
                │   35 │   )                                                                                                              │                
                │   36                                                                                                                    │                
                │                                                                                                                         │                
                │ /home/data/.local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py:877 in run_sync_in_worker_thread              │                
                │                                                                                                                         │                
                │    874 │   │   │   context = copy_context()                                                                             │                
                │    875 │   │   │   context.run(sniffio.current_async_library_cvar.set, None)                                            │                
                │    876 │   │   │   worker.queue.put_nowait((context, func, args, future))                                               │                
                │ ❱  877 │   │   │   return await future                                                                                  │                
                │    878                                                                                                                  │                
                │    879                                                                                                                  │                
                │    880 def run_sync_from_thread(                                                                                        │                
                │                                                                                                                         │                
                │ /home/data/.local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py:807 in run                                    │                
                │                                                                                                                         │                
                │    804 │   │   │   │   │   result = None                                                                                │                
                │    805 │   │   │   │   │   exception: BaseException | None = None                                                       │                
                │    806 │   │   │   │   │   try:                                                                                         │                
                │ ❱  807 │   │   │   │   │   │   result = context.run(func, *args)                                                        │                
                │    808 │   │   │   │   │   except BaseException as exc:                                                                 │                
                │    809 │   │   │   │   │   │   exception = exc                                                                          │                
                │    810                                                                                                                  │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/khoj/routers/api.py:623 in chat_history                      │                
                │                                                                                                                         │                
                │   620 │   referer: Optional[str] = Header(None),                                                                        │                
                │   621 │   host: Optional[str] = Header(None),                                                                           │                
                │   622 ):                                                                                                                │                
                │ ❱ 623 │   perform_chat_checks()                                                                                         │                
                │   624 │                                                                                                                 │                
                │   625 │   # Load Conversation History                                                                                   │                
                │   626 │   meta_log = {}                                                                                                 │                
                │                                                                                                                         │                
                │ /home/data/anaconda3/envs/KHOJ/lib/python3.8/site-packages/khoj/routers/helpers.py:18 in perform_chat_checks            │                
                │                                                                                                                         │                
                │    15                                                                                                                   │                
                │    16                                                                                                                   │                
                │    17 def perform_chat_checks():                                                                                        │                
                │ ❱  18 │   if state.processor_config.conversation and (                                                                  │                
                │    19 │   │   state.processor_config.conversation.openai_model                                                          │                
                │    20 │   │   or state.processor_config.conversation.gpt4all_model.loaded_model                                         │                
                │    21 │   ):                                                                                                            │                
                ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                
                AttributeError: 'NoneType' object has no attribute 'conversation'                                                                          
sabaimran commented 1 year ago

Hmm, it looks like you're on an older version of khoj. Could you tell me the output of khoj --version?

And could you try running pip install --upgrade khoj and try restarting it?

Gangxin-Li commented 1 year ago

Here is the command result.

(base) VM:~$ conda activate KHOJ
(KHOJ) VM:~$ khoj --version
0.10.0
(KHOJ) VM:~$ pip install --upgrade khoj
ERROR: Could not find a version that satisfies the requirement khoj (from versions: none)
ERROR: No matching distribution found for khoj
(KHOJ) VM:~$ pip install --upgrade khoj
ERROR: Could not find a version that satisfies the requirement khoj (from versions: none)
ERROR: No matching distribution found for khoj

I do remember I upgrade it before, and due to some reasons, I need to create a new environment to install it again. And there are some packages conflict.

sabaimran commented 1 year ago

Hmm, that's a bit strange. Do you mind sharing the following from your VM. I'm guessing there's some conflict between the dependency and the builds supported by the pypi package:

Gangxin-Li commented 1 year ago

Yeah, I wish so Here is my VM environment Operating system: Ubuntu 20.04.6 LTS Architecture: Not sure, but here is my VM: Standard NC4as T4 v3 (4 vcpus, 28 GiB memory) Python version: 3.8.17 pip version: 23.2.1

debanjum commented 1 year ago

If you're trying to run Khoj on a VM, can you use Ubuntu 22.04 with Python 3.9 or above to install Khoj?

Khoj doesn't support Python 3.8 anymore and folks have had trouble using Khoj on Ubuntu 20.04 (which needs to be investigated)

debanjum commented 1 year ago

Closing this as Khoj requires Python version >=3.9. Feel free to re-open if you're unable to run Khoj with a newer python