Closed zooninja closed 9 months ago
tomasonjo
commented
9 months ago You need to include the following env variables besides the OPENAI_API_KEY in order to use OpenAI LLM and embedding model
OPENAI_API_KEY=sk-...
LLM=gpt-3.5
EMBEDDING_MODEL=openai
zooninja
commented
9 months ago Thank you have done as suggested but my issue evolved, Please find output below as well as my .env configuration Looks like some kind of connectivity issue now. Please advise. Running Docker on rocky 9.2
OPENAI_API_KEY=sk-Ek**** OLLAMA_BASE_URL=http://host.docker.internal:11434 NEO4J_URI=neo4j://localhost:7687 NEO4J_USERNAME=neo4j NEO4J_PASSWORD=password LLM=gpt-3.5 EMBEDDING_MODEL=openai
LANGCHAIN_ENDPOINT="https://api.smith.langchain.com" LANGCHAIN_TRACING_V2=true # false LANGCHAIN_PROJECT=#your-project-name LANGCHAIN_API_KEY=sk-Ek****
Attaching to genai-stack-bot-1, genai-stack-database-1, genai-stack-loader-1, genai-stack-pdf_bot-1, genai-stack-pull-model-1
genai-stack-pull-model-1 | pulling ollama model gpt-3.5 using http://host.docker.internal:11434
genai-stack-pull-model-1 | OLLAMA model only pulled if both LLM and OLLAMA_BASE_URL are set and the LLM model is not gpt
genai-stack-pull-model-1 exited with code 0
genai-stack-database-1 | Installing Plugin 'apoc' from /var/lib/neo4j/labs/apoc-*-core.jar to /var/lib/neo4j/plugins/apoc.jar
genai-stack-database-1 | Applying default values for plugin apoc to neo4j.conf
genai-stack-database-1 | Changed password for user 'neo4j'. IMPORTANT: this change will only take effect if performed before the database is started for the first time.
genai-stack-database-1 | 2023-10-06 10:12:48.268+0000 INFO Starting...
genai-stack-database-1 | 2023-10-06 10:12:49.613+0000 INFO This instance is ServerId{df0d3820} (df0d3820-fdb9-45fa-b45b-4f519b22dd1d)
genai-stack-database-1 | 2023-10-06 10:12:51.123+0000 INFO ======== Neo4j 5.11.0 ========
genai-stack-database-1 | 2023-10-06 10:12:57.984+0000 INFO Bolt enabled on 0.0.0.0:7687.
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.Server - jetty-10.0.15; built: 2023-04-11T17:25:14.480Z; git: 68017dbd00236bb7e187330d7585a059610f661d; jvm 17.0.8.1+1
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.h.MovedContextHandler@52999774{/,null,AVAILABLE}
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.session.DefaultSessionIdManager - Session workerName=node0
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@17e9e99{/db,null,AVAILABLE}
genai-stack-database-1 | [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor - NO JSP Support for /browser, did not find org.eclipse.jetty.jsp.JettyJspServlet
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@2639ec96{/browser,jar:file:/var/lib/neo4j/lib/neo4j-browser-5.11.0.jar!/browser,AVAILABLE}
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@1a0231f9{/,null,AVAILABLE}
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.AbstractConnector - Started http@60296d11{HTTP/1.1, (http/1.1)}{0.0.0.0:7474}
genai-stack-database-1 | [main] INFO org.eclipse.jetty.server.Server - Started Server@2ef4e7f3{STARTING}[10.0.15,sto=0] @12278ms
genai-stack-database-1 | 2023-10-06 10:12:59.151+0000 INFO Remote interface available at http://localhost:7474/
genai-stack-database-1 | 2023-10-06 10:12:59.156+0000 INFO id: 7C5A709C7738816AF7780AF990A46F4BA4FD1625930DC3A7C06E77B69F0D2C88
genai-stack-database-1 | 2023-10-06 10:12:59.157+0000 INFO name: system
genai-stack-database-1 | 2023-10-06 10:12:59.157+0000 INFO creationDate: 2023-10-06T09:38:08.592Z
genai-stack-database-1 | 2023-10-06 10:12:59.158+0000 INFO Started.
genai-stack-bot-1 |
genai-stack-bot-1 | Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.
genai-stack-bot-1 |
genai-stack-pdf_bot-1 |
genai-stack-pdf_bot-1 | Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.
genai-stack-pdf_bot-1 |
genai-stack-loader-1 |
genai-stack-loader-1 | Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.
genai-stack-loader-1 |
genai-stack-bot-1 |
genai-stack-bot-1 | You can now view your Streamlit app in your browser.
genai-stack-bot-1 |
genai-stack-bot-1 | URL: http://0.0.0.0:8501
genai-stack-pdf_bot-1 |
genai-stack-pdf_bot-1 | You can now view your Streamlit app in your browser.
genai-stack-pdf_bot-1 |
genai-stack-pdf_bot-1 | URL: http://0.0.0.0:8503
genai-stack-pdf_bot-1 |
genai-stack-bot-1 |
genai-stack-loader-1 |
genai-stack-loader-1 | You can now view your Streamlit app in your browser.
genai-stack-loader-1 |
genai-stack-loader-1 | URL: http://0.0.0.0:8502
genai-stack-loader-1 |
genai-stack-loader-1 | 2023-10-06 10:14:04.182 Embedding: Using OpenAI
genai-stack-loader-1 | Unable to retrieve routing information
genai-stack-loader-1 | 2023-10-06 10:14:04.336 Uncaught app exception
genai-stack-loader-1 | Traceback (most recent call last):
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/langchain/graphs/neo4j_graph.py", line 52, in init
genai-stack-loader-1 | self._driver.verify_connectivity()
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/neo4j/_sync/driver.py", line 985, in verify_connectivity
genai-stack-loader-1 | self._get_server_info(session_config)
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/neo4j/_sync/driver.py", line 1200, in _get_server_info
genai-stack-loader-1 | return session._get_server_info()
genai-stack-loader-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/neo4j/_sync/work/session.py", line 175, in _get_server_info
genai-stack-loader-1 | self._connect(READ_ACCESS, liveness_check_timeout=0)
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/neo4j/_sync/work/session.py", line 133, in _connect
genai-stack-loader-1 | super()._connect(
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/neo4j/_sync/work/workspace.py", line 161, in _connect
genai-stack-loader-1 | self._pool.update_routing_table(
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/neo4j/_sync/io/_pool.py", line 804, in update_routing_table
genai-stack-loader-1 | raise ServiceUnavailable("Unable to retrieve routing information")
genai-stack-loader-1 | neo4j.exceptions.ServiceUnavailable: Unable to retrieve routing information
genai-stack-loader-1 |
genai-stack-loader-1 | During handling of the above exception, another exception occurred:
genai-stack-loader-1 |
genai-stack-loader-1 | Traceback (most recent call last):
genai-stack-loader-1 | File "/usr/local/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 541, in _run_script
genai-stack-loader-1 | exec(code, module.dict)
genai-stack-loader-1 | File "/app/loader.py", line 30, in
tomasonjo
commented
9 months ago Try using
neo4j://database:7687
As neo4j_uri variable
zooninja
commented
9 months ago Thank you I did that and when I docker compose up --build again I see containers are starting. the loader starts last, hangs for a while and then fails like this:
when I inspect the container I get this output
docker inspect 9381a31a2d53 [ { "Id": "9381a31a2d53805246e5c22262fad3d2cde3ca90b55124267ab7b173c44463a4", "Created": "2023-10-06T11:08:47.080540504Z", "Path": "streamlit", "Args": [ "run", "loader.py", "--server.port=8502", "--server.address=0.0.0.0" ], "State": { "Status": "running", "Running": true, "Paused": false, "Restarting": false, "OOMKilled": false, "Dead": false, "Pid": 339091, "ExitCode": 0, "Error": "", "StartedAt": "2023-10-06T11:09:10.310846867Z", "FinishedAt": "0001-01-01T00:00:00Z", "Health": { "Status": "unhealthy", "FailingStreak": 9, "Log": [ { "Start": "2023-10-06T13:11:41.114939255+02:00", "End": "2023-10-06T13:11:41.347251888+02:00", "ExitCode": 7, "Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\ncurl: (7) Failed to connect to localhost port 8501 after 0 ms: Couldn't connect to server\n" }, { "Start": "2023-10-06T13:12:11.351163636+02:00", "End": "2023-10-06T13:12:11.601723201+02:00", "ExitCode": 7, "Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\ncurl: (7) Failed to connect to localhost port 8501 after 4 ms: Couldn't connect to server\n" }, { "Start": "2023-10-06T13:12:41.611326305+02:00", "End": "2023-10-06T13:12:41.815902905+02:00", "ExitCode": 7, "Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\ncurl: (7) Failed to connect to localhost port 8501 after 1 ms: Couldn't connect to server\n" }, { "Start": "2023-10-06T13:13:11.819742775+02:00", "End": "2023-10-06T13:13:11.997058497+02:00", "ExitCode": 7, "Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\ncurl: (7) Failed to connect to localhost port 8501 after 0 ms: Couldn't connect to server\n" }, { "Start": "2023-10-06T13:13:41.999843634+02:00", "End": "2023-10-06T13:13:42.231723833+02:00", "ExitCode": 7, "Output": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\ncurl: (7) Failed to connect to localhost port 8501 after 0 ms: Couldn't connect to server\n" } ] } },
Output from docker compose command:
genai-stack-loader-1 | 2023-10-06 11:02:02.799 Embedding: Using OpenAI genai-stack-bot-1 | Stopping... genai-stack-loader-1 | Stopping... genai-stack-pdf_bot-1 | Stopping... genai-stack-bot-1 exited with code 0 genai-stack-pdf_bot-1 exited with code 0 genai-stack-pdf_bot-1 exited with code 0 genai-stack-loader-1 exited with code 0 genai-stack-loader-1 exited with code 0 genai-stack-database-1 | 2023-10-06 11:08:11.216+0000 INFO Neo4j Server shutdown initiated by request genai-stack-database-1 | 2023-10-06 11:08:11.218+0000 INFO Stopping... genai-stack-database-1 | [Thread-0] INFO org.eclipse.jetty.server.Server - Stopped Server@7e58471{STOPPING}[10.0.15,sto=0] genai-stack-database-1 | [Thread-0] INFO org.eclipse.jetty.server.AbstractConnector - Stopped http@2492f6fb{HTTP/1.1, (http/1.1)}{0.0.0.0:7474} genai-stack-database-1 | [Thread-0] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@7a4764bb{/,null,STOPPED} genai-stack-database-1 | [Thread-0] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@6e14a376{/browser,jar:file:/var/lib/neo4j/lib/neo4j-browser-5.11.0.jar!/browser,STOPPED} genai-stack-database-1 | [Thread-0] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@42542698{/db,null,STOPPED} genai-stack-database-1 | [Thread-0] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.h.MovedContextHandler@5e9affdc{/,null,STOPPED} genai-stack-database-1 | 2023-10-06 11:08:16.576+0000 INFO Stopped. genai-stack-database-1 exited with code 0
zooninja
commented
9 months ago I got my issue solved. Thanks to @tomasonjo for his help.
Please advise the correct configuration for gpt 3.5 usage as I keep getting messages related to Ollama and python related crash outputs when I deploy the bots. gpt 3.5 API key is in the .env file. The only web that starts correctly is the pdf one but when I upload a file it crashes too. I can provide evidence a little bit later.