search

serge-chat / serge

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.
https://serge.chat
Apache License 2.0
5.64k stars 400 forks source link

Web page just gives 500 error #29

Closed arjones85 closed 1 year ago

arjones85 commented 1 year ago

Trying to get this running, but when I visit port 8008 I just get a "500 Internal Error" page. Are you able to help?

Logs from the containers:

[root@box serge]# docker compose up 
[+] Running 5/5
 ⠿ Network serge_default      Created                                                                                                                                   0.1s
 ⠿ Container serge-web-1      Created                                                                                                                                   1.2s
 ⠿ Container serge-nginx-1    Created                                                                                                                                   0.1s
 ⠿ Container serge-mongodb-1  Created                                                                                                                                   0.1s
 ⠿ Container serge-api-1      Created                                                                                                                                   0.0s
Attaching to serge-api-1, serge-mongodb-1, serge-nginx-1, serge-web-1
serge-nginx-1    | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
serge-nginx-1    | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
serge-nginx-1    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
serge-nginx-1    | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
serge-nginx-1    | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
serge-nginx-1    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
serge-nginx-1    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
serge-nginx-1    | /docker-entrypoint.sh: Configuration complete; ready for start up
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: using the "epoll" event method
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: nginx/1.23.3
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: built by gcc 12.2.1 20220924 (Alpine 12.2.1_git20220924-r4) 
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: OS: Linux 6.2.2-1.el8.elrepo.x86_64
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker processes
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 29
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 30
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 31
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 32
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 33
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 34
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 35
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 36
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 37
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 38
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 39
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 40
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 41
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 42
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 43
serge-nginx-1    | 2023/03/23 13:23:34 [notice] 1#1: start worker process 44
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.927+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"-","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.928+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.928+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.928+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.928+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.928+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.928+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"600d9ce93974"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.4","gitVersion":"44ff59461c1353638a71e710f385a566bcd2f547","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2204","distarch":"x86_64","target_arch":"x86_64"}}}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"22.04"}}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"}}}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEngine":"wiredTiger"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"STORAGE",  "id":22297,   "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:34.929+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=6935M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],"}}
serge-api-1      | INFO:     Will watch for changes in these directories: ['/usr/src/app']
serge-api-1      | INFO:     Uvicorn running on http://0.0.0.0:9124 (Press CTRL+C to quit)
serge-api-1      | INFO:     Started reloader process [1] using WatchFiles
serge-web-1      | 
serge-web-1      | > web@0.0.1 dev
serge-web-1      | > vite dev --host 0.0.0.0 --port 9123
serge-web-1      | 
serge-api-1      | Process SpawnProcess-1:
serge-api-1      | Traceback (most recent call last):
serge-api-1      |   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
serge-api-1      |     self.run()
serge-api-1      |   File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
serge-api-1      |     self._target(*self._args, **self._kwargs)
serge-api-1      |   File "/usr/local/lib/python3.8/dist-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
serge-api-1      |     target(sockets=sockets)
serge-api-1      |   File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 59, in run
serge-api-1      |     return asyncio.run(self.serve(sockets=sockets))
serge-api-1      |   File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
serge-api-1      |     return loop.run_until_complete(main)
serge-api-1      |   File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
serge-api-1      |   File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 66, in serve
serge-api-1      |     config.load()
serge-api-1      |   File "/usr/local/lib/python3.8/dist-packages/uvicorn/config.py", line 471, in load
serge-api-1      |     self.loaded_app = import_from_string(self.app)
serge-api-1      |   File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 24, in import_from_string
serge-api-1      |     raise exc from None
serge-api-1      |   File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 21, in import_from_string
serge-api-1      |     module = importlib.import_module(module_str)
serge-api-1      |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
serge-api-1      |     return _bootstrap._gcd_import(name[level:], package, level)
serge-api-1      |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
serge-api-1      |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
serge-api-1      |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
serge-api-1      |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
serge-api-1      |   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
serge-api-1      |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
serge-api-1      |   File "/usr/src/app/main.py", line 4, in <module>
serge-api-1      |     from typing import Annotated
serge-api-1      | ImportError: cannot import name 'Annotated' from 'typing' (/usr/lib/python3.8/typing.py)
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.483+00:00"},"s":"I",  "c":"STORAGE",  "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":554}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.483+00:00"},"s":"I",  "c":"RECOVERY", "id":23987,   "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.490+00:00"},"s":"W",  "c":"CONTROL",  "id":22120,   "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.490+00:00"},"s":"W",  "c":"CONTROL",  "id":22178,   "ctx":"initandlisten","msg":"/sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'","tags":["startupWarnings"]}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.490+00:00"},"s":"W",  "c":"CONTROL",  "id":5123300, "ctx":"initandlisten","msg":"vm.max_map_count is too low","attr":{"currentValue":65530,"recommendedMinimum":1677720,"maxConns":838860},"tags":["startupWarnings"]}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.492+00:00"},"s":"I",  "c":"NETWORK",  "id":4915702, "ctx":"initandlisten","msg":"Updated wire specification","attr":{"oldSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true},"newSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":17,"maxWireVersion":17},"outgoing":{"minWireVersion":17,"maxWireVersion":17},"isInternalClient":true}}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.492+00:00"},"s":"I",  "c":"REPL",     "id":5853300, "ctx":"initandlisten","msg":"current featureCompatibilityVersion value","attr":{"featureCompatibilityVersion":"6.0","context":"startup"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.492+00:00"},"s":"I",  "c":"STORAGE",  "id":5071100, "ctx":"initandlisten","msg":"Clearing temp directory"}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.492+00:00"},"s":"I",  "c":"CONTROL",  "id":20536,   "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.493+00:00"},"s":"I",  "c":"FTDC",     "id":20625,   "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/db/diagnostic.data"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.500+00:00"},"s":"I",  "c":"REPL",     "id":6015317, "ctx":"initandlisten","msg":"Setting new configuration state","attr":{"newState":"ConfigReplicationDisabled","oldState":"ConfigPreStart"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.500+00:00"},"s":"I",  "c":"STORAGE",  "id":22262,   "ctx":"initandlisten","msg":"Timestamp monitor starting"}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.501+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.501+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"0.0.0.0"}}
serge-mongodb-1  | {"t":{"$date":"2023-03-23T13:23:35.501+00:00"},"s":"I",  "c":"NETWORK",  "id":23016,   "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
serge-web-1      | 
serge-web-1      | Forced re-optimization of dependencies
serge-web-1      | 
serge-web-1      |   VITE v4.2.0  ready in 556 ms
serge-web-1      | 
serge-web-1      |   ➜  Local:   http://localhost:9123/
serge-web-1      |   ➜  Network: http://172.23.0.3:9123/
serge-web-1      | TypeError: fetch failed
serge-web-1      |     at fetch (/usr/src/app/node_modules/undici/index.js:109:13)
serge-web-1      |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
serge-web-1      |     at async Object.eval [as fetch] (/node_modules/@sveltejs/kit/src/runtime/server/fetch.js:27:10)
serge-web-1      |     at async eval (/node_modules/@sveltejs/kit/src/runtime/server/page/load_data.js:195:18)
serge-web-1      |     at async load (+layout.ts:11:12)
serge-web-1      |     at async Module.load_data (/node_modules/@sveltejs/kit/src/runtime/server/page/load_data.js:162:17)
serge-web-1      |     at async eval (/node_modules/@sveltejs/kit/src/runtime/server/page/index.js:169:13)
serge-nginx-1    | 10.150.1.5 - - [23/Mar/2023:13:23:40 +0000] "GET / HTTP/1.1" 500 1029 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-web-1      | TypeError: fetch failed
serge-web-1      |     at fetch (/usr/src/app/node_modules/undici/index.js:109:13)
serge-web-1      |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
serge-web-1      |     at async Object.eval [as fetch] (/node_modules/@sveltejs/kit/src/runtime/server/fetch.js:27:10)
serge-web-1      |     at async eval (/node_modules/@sveltejs/kit/src/runtime/server/page/load_data.js:195:18)
serge-web-1      |     at async load (+layout.ts:11:12)
serge-web-1      |     at async Module.load_data (/node_modules/@sveltejs/kit/src/runtime/server/page/load_data.js:162:17)
serge-web-1      |     at async Module.respond_with_error (/node_modules/@sveltejs/kit/src/runtime/server/page/respond_with_error.js:52:17)
serge-web-1      |     at async resolve (/node_modules/@sveltejs/kit/src/runtime/server/respond.js:388:12)
serge-web-1      |     at async Module.respond (/node_modules/@sveltejs/kit/src/runtime/server/respond.js:240:20)
serge-web-1      |     at async file:///usr/src/app/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:505:22
serge-nginx-1    | 10.150.1.5 - - [23/Mar/2023:13:23:40 +0000] "GET /favicon.ico HTTP/1.1" 500 1019 "http://192.168.1.110:8008/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
nsarrazin commented 1 year ago

Looks like you're running python 3.8 for some reason which doesn't support the typings we use.

Can you give me some details about your OS, and your docker version ?

arjones85 commented 1 year ago

Sure - Rocky 8.7 and Docker CE 23

arjones85 commented 1 year ago

Also, I'm actually running Python 3.6. The errors about 3.8 is the version of Python inside the container.

Python 3.9 is available in Rocky. If you think the host python version is the issue, I'll upgrade and try it.

Edit: Made Python 3.9 default python, tried it again, same errors unfortunately.

OilProducts commented 1 year ago

Its a problem with the python version inside the container. Simply running docker compose up builds a container with 3.8. not sure how anyone is getting it to work...

Edit: the "docker:latest" on my machine was 2 years old, pulling a new one works fine.

arjones85 commented 1 year ago

the "docker:latest" on my machine was 2 years old, pulling a new one works fine.

Sorry - Pulling a new one of what? I went through and did a docker pull of nginx:alpine, ubuntu:latest, node:latest, and docker:latest and it hasn't made any difference for me. Still the same error, it's still running 3.8 in the containers.

OilProducts commented 1 year ago

Sorry I mis typed. "ubuntu:latest" was the problem.

arjones85 commented 1 year ago

Thanks! I ended up stopping all my other containers, pruning all images and the containers themselves, then doing a pull. That seems to have resolved that error.

Now, when I try to send anything to it, it is erroring out with:

serge-nginx-1    | 10.150.1.5 - - [23/Mar/2023:17:11:35 +0000] "GET /__data.json?x-sveltekit-invalidated=_1 HTTP/1.1" 200 82 "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1      | INFO:     172.27.0.3:47002 - "GET /models HTTP/1.1" 200 OK
serge-nginx-1    | 10.150.1.5 - - [23/Mar/2023:17:11:35 +0000] "GET /api/models HTTP/1.1" 200 2 "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1      | INFO:     172.27.0.3:47004 - "GET /chat/9b82505e-f78d-4fe1-97a3-1323117cc6de HTTP/1.1" 200 OK
serge-nginx-1    | 10.150.1.5 - - [23/Mar/2023:17:11:36 +0000] "GET /api/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de HTTP/1.1" 200 497 "http://192.168.1.110:8008/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1      | INFO:     172.27.0.3:53868 - "GET /chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test HTTP/1.1" 200 OK
serge-nginx-1    | 2023/03/23 17:11:43 [error] 34#34: *78 upstream prematurely closed connection while reading upstream, client: 10.150.1.5, server: localhost, request: "GET /api/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test HTTP/1.1", upstream: "http://172.27.0.5:9124/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test", host: "192.168.1.110:8008", referrer: "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de"
serge-api-1      | ERROR:    Exception in ASGI application
serge-api-1      | Traceback (most recent call last):
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 436, in run_asgi
serge-api-1      |     result = await app(  # type: ignore[func-returns-value]
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
serge-api-1      |     return await self.app(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 276, in __call__
serge-api-1      |     await super().__call__(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 122, in __call__
serge-api-1      |     await self.middleware_stack(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 184, in __call__
serge-api-1      |     raise exc
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 162, in __call__
serge-api-1      |     await self.app(scope, receive, _send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 84, in __call__
serge-api-1      |     await self.app(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
serge-api-1      |     raise exc
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
serge-api-1      |     await self.app(scope, receive, sender)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
serge-api-1      |     raise e
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
serge-api-1      |     await self.app(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 718, in __call__
serge-nginx-1    | 10.150.1.5 - - [23/Mar/2023:17:11:43 +0000] "GET /api/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de/question?prompt=Test HTTP/1.1" 200 0 "http://192.168.1.110:8008/chat/9b82505e-f78d-4fe1-97a3-1323117cc6de" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
serge-api-1      |     await route.handle(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 276, in handle
serge-api-1      |     await self.app(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 69, in app
serge-api-1      |     await response(scope, receive, send)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/sse_starlette/sse.py", line 227, in __call__
serge-api-1      |     async with anyio.create_task_group() as task_group:
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
serge-api-1      |     raise exceptions[0]
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/sse_starlette/sse.py", line 230, in wrap
serge-api-1      |     await func()
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/sse_starlette/sse.py", line 219, in stream_response
serge-api-1      |     async for data in self.body_iterator:
serge-api-1      |   File "/usr/src/app/main.py", line 161, in event_generator
serge-api-1      |     async for output in generate(
serge-api-1      |   File "/usr/src/app/utils/generate.py", line 65, in generate
serge-api-1      |     raise ValueError(error_output.decode("utf-8"))
serge-api-1      | ValueError: main: seed = 1679591503
serge-api-1      | llama_model_load: loading model from '/usr/src/app/weights/ggml-alpaca-7B-q4_0.bin' - please wait ...
serge-api-1      | llama_model_load: failed to open '/usr/src/app/weights/ggml-alpaca-7B-q4_0.bin'
serge-api-1      | llama_init_from_file: failed to load model
serge-api-1      | main: error: failed to load model '/usr/src/app/weights/ggml-alpaca-7B-q4_0.bin'

I think I'll wait a bit before trying it out again, let this project get industrialized a bit :-)

nsarrazin commented 1 year ago

I've fixed the dependencies in my latest commits. Feel free to pull the latest from main and try again, hopefully it works better now.

pabl-o-ce commented 1 year ago

Hi @nsarrazin , I'm using at host Python 3.10.10 get same 502 error at browser and in the logs I found this errors

serge-api-1      | INFO:     Started reloader process [1] using WatchFiles
serge-api-1      | INFO:     Started server process [8]
serge-api-1      | INFO:     Waiting for application startup.
serge-api-1      | INFO:        main    initializing database connection
serge-api-1      | ERROR:    Traceback (most recent call last):
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 677, in lifespan
serge-api-1      |     async with self.lifespan_context(app) as maybe_state:
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 566, in __aenter__
serge-api-1      |     await self._router.startup()
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 654, in startup
serge-api-1      |     await handler()
serge-api-1      |   File "/usr/src/app/main.py", line 96, in start_database
serge-api-1      |     await initiate_database()
serge-api-1      |   File "/usr/src/app/utils/initiate_database.py", line 25, in initiate_database
serge-api-1      |     await init_beanie(
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 528, in init_beanie
serge-api-1      |     await Initializer(
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 89, in __await__
serge-api-1      |     yield from self.init_class(model).__await__()
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 498, in init_class
serge-api-1      |     await self.init_document(cls)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/beanie/odm/utils/init.py", line 322, in init_document
serge-api-1      |     build_info = await self.database.command({"buildInfo": 1})
serge-api-1      |   File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
serge-api-1      |     result = self.fn(*self.args, **self.kwargs)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/_csot.py", line 105, in csot_wrapper
serge-api-1      |     return func(self, *args, **kwargs)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/database.py", line 805, in command
serge-api-1      |     with self.__client._socket_for_reads(read_preference, session) as (
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/mongo_client.py", line 1296, in _socket_for_reads
serge-api-1      |     server = self._select_server(read_preference, session)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/mongo_client.py", line 1257, in _select_server
serge-api-1      |     server = topology.select_server(server_selector)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 272, in select_server
serge-api-1      |     server = self._select_server(selector, server_selection_timeout, address)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 261, in _select_server
serge-api-1      |     servers = self.select_servers(selector, server_selection_timeout, address)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 223, in select_servers
serge-api-1      |     server_descriptions = self._select_servers_loop(selector, server_timeout, address)
serge-api-1      |   File "/usr/local/lib/python3.10/dist-packages/pymongo/topology.py", line 238, in _select_servers_loop
serge-api-1      |     raise ServerSelectionTimeoutError(
serge-api-1      | pymongo.errors.ServerSelectionTimeoutError: mongodb:27017: [Errno -2] Name or service not known, Timeout: 30s, Topology Description: <TopologyDescription id: 641d1cdd03bb7fd9fe7b2a4e, topology_type: Unknown, servers: [<ServerDescription ('mongodb', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('mongodb:27017: [Errno -2] Name or service not known')>]>
serge-api-1      |
serge-api-1      | ERROR:    Application startup failed. Exiting.

any tip? what version you need of python to work it?

I'm with the latest commit

Psd: Amazing work

pabl-o-ce commented 1 year ago

forget it I resolved... I was trying to skip nginx and somehow it get very messy

voronind-com commented 1 year ago

I get the same TypeError: fetch failed on my server with nginx proxy with TLS. same docker image on my workstation works (no nginx, direct access to 8008 port). waiting for proxy support.

pabl-o-ce commented 1 year ago

@cakee-ru i make it work for the current version serge are with this config on nginx

server {
  listen [::]:443 ssl http2;
  listen 443 ssl http2;

  # The host name to respond to
  server_name ai.yourdomain.com;

  ...your config...

  # Path for static files
  location / {
    proxy_pass http://xxx.xxx.xxx.xxx:8008/;
    proxy_redirect off;
  }

}

server {
  listen [::]:80;
  listen 80;

  server_name ai.yourdomain.com;

  # (1)
  return 301 https://$host$request_uri;
  # return 444;
}

xxx.xxx.xxx.xxx is the ip your are using local o docker network

voronind-com commented 1 year ago

@pabl-o-ce the key part is proxy_redirect off;? cause I have pretty much the same basic config.

pabl-o-ce commented 1 year ago

you can skip it.. i was messing around the best way to make it work.. the important part right now (but it will resolve soon #84 84) is to put / and the end of proxy_pass http://ip:port/... I'm not into svelte but it work for proxy nginx with a domain

gaby commented 1 year ago

@arjones85 Still having issues with this?