search

opea-project / GenAIExamples

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.
https://opea.dev
Apache License 2.0
252 stars 174 forks source link

ChatQnA - Pre-built containers don't work #460

Closed wsfowler closed 2 months ago

wsfowler commented 3 months ago

I'm trying to run the example using ChatQnA/docker/xeon/docker_compose_vllm.yaml and pulling the containers from Docker Hub instead of building the latest versions and I'm getting errors.

The containers all start up ok:

root@ip-172-31-29-39:/opt/GenAIExamples/ChatQnA/docker/xeon# docker ps
CONTAINER ID   IMAGE                                                   COMMAND                  CREATED          STATUS          PORTS                                                                                  NAMES
01e761700075   opea/chatqna-ui:latest                                  "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:5173->5173/tcp, :::5173->5173/tcp                                              chatqna-xeon-ui-server
7147fa77ff44   opea/chatqna-conversation-ui:latest                     "nginx -g 'daemon of…"   16 minutes ago   Up 16 minutes   0.0.0.0:5174->80/tcp, :::5174->80/tcp                                                  chatqna-xeon-conversation-ui-server
77ef4f7550d5   opea/chatqna:latest                                     "python chatqna.py"      16 minutes ago   Up 16 minutes   0.0.0.0:8888->8888/tcp, :::8888->8888/tcp                                              chatqna-xeon-backend-server
3601a3d8d5b3   opea/dataprep-redis:latest                              "python prepare_doc_…"   16 minutes ago   Up 16 minutes   0.0.0.0:6007-6009->6007-6009/tcp, :::6007-6009->6007-6009/tcp                          dataprep-redis-server
6ea0e6296b62   opea/retriever-redis:latest                             "/home/user/comps/re…"   16 minutes ago   Up 16 minutes   0.0.0.0:7000->7000/tcp, :::7000->7000/tcp                                              retriever-redis-server
c097372544ea   opea/reranking-tei:latest                               "python reranking_te…"   16 minutes ago   Up 16 minutes   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp                                              reranking-tei-xeon-server
6e7f2e82fc33   opea/llm-vllm:latest                                    "bash entrypoint.sh"     16 minutes ago   Up 16 minutes   0.0.0.0:9000->9000/tcp, :::9000->9000/tcp                                              llm-vllm-server
2fde8265a23b   opea/embedding-tei:latest                               "python embedding_te…"   16 minutes ago   Up 16 minutes   0.0.0.0:6000->6000/tcp, :::6000->6000/tcp                                              embedding-tei-server
716509f4e2e5   ghcr.io/huggingface/text-embeddings-inference:cpu-1.2   "text-embeddings-rou…"   16 minutes ago   Up 16 minutes   0.0.0.0:8808->80/tcp, :::8808->80/tcp                                                  tei-reranking-server
303677d1631c   redis/redis-stack:7.2.0-v9                              "/entrypoint.sh"         16 minutes ago   Up 16 minutes   0.0.0.0:6379->6379/tcp, :::6379->6379/tcp, 0.0.0.0:8001->8001/tcp, :::8001->8001/tcp   redis-vector-db
aa8d53791788   vllm:cpu                                                "python3 -m vllm.ent…"   16 minutes ago   Up 16 minutes   0.0.0.0:9009->80/tcp, :::9009->80/tcp                                                  vllm-service
740e5445d19d   ghcr.io/huggingface/text-embeddings-inference:cpu-1.2   "text-embeddings-rou…"   16 minutes ago   Up 16 minutes   0.0.0.0:6006->80/tcp, :::6006->80/tcp                                                  tei-embedding-server

When I try to use the conversation ui (IP:5174) I get the following error when I try to upload a document in the browser:

POST http://54.221.135.195:5174/ 405 (Not Allowed)

This is the error from the conversation-ui container:

192.55.55.48 - - [25/Jul/2024:17:15:59 +0000] "POST / HTTP/1.1" 413 585 "http://54.221.135.195:5174/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" "-"
192.55.55.48 - - [25/Jul/2024:17:16:17 +0000] "POST / HTTP/1.1" 405 559 "http://54.221.135.195:5174/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" "-"

I see this error for the reranking container:

root@ip-172-31-29-39:/opt/GenAIExamples/ChatQnA/docker/xeon# docker logs 9c633102f747
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:161: UserWarning: Field "model_name_or_path" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[2024-07-25 16:40:40,159] [    INFO] - CORS is enabled.
[2024-07-25 16:40:40,160] [    INFO] - Setting up HTTP server
[2024-07-25 16:40:40,161] [    INFO] - Uvicorn server setup on port 8000
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
[2024-07-25 16:40:40,171] [    INFO] - HTTP server setup successful
WARNING:  Invalid HTTP request received.

The embedding-tei container talks about Gaudi, which seems weird:

root@ip-172-31-29-39:/opt/GenAIExamples/ChatQnA/docker/xeon# docker logs 2fde8265a23b
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:149: UserWarning: Field "model_name_or_path" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[2024-07-25 16:57:45,598] [    INFO] - CORS is enabled.
[2024-07-25 16:57:45,599] [    INFO] - Setting up HTTP server
[2024-07-25 16:57:45,599] [    INFO] - Uvicorn server setup on port 6000
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:6000 (Press CTRL+C to quit)
[2024-07-25 16:57:45,696] [    INFO] - HTTP server setup successful
TEI Gaudi Embedding initialized.

None of the other containers have errors.

wsfowler commented 3 months ago

This doesn't fix the issue, but you might take a look at this issue: https://github.com/opea-project/GenAIComps/issues/357

huiyan2021 commented 2 months ago

I can reproduce the issue, looking into it

WenjiaoYue commented 2 months ago

Hello @jaswanth8888,

This issue is related to the React code. It seems there is a problem with your Dockerfile; it does not correctly store and run in the Docker image. As a result, while the front-end page loads properly, the corresponding API cannot be found when making requests. Please correct this issue, test the Docker setup locally to ensure it works end-to-end, and then upload it again. Thank you. (As shown in the image, the front-end requests are not correctly calling the API to access the back-end.) image

huiyan2021 commented 2 months ago

Hi @jaswanth8888, any updates?

jaswanth8888 commented 2 months ago

Hi @huiyan2021 this issue has been fixed in https://github.com/opea-project/GenAIExamples/pull/590. this would let the container pic environment variables during runtime