search

opea-project / GenAIExamples

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.
https://opea.dev
Apache License 2.0
174 stars 83 forks source link

ChatQnA readme issues #87

Closed tbykowsk closed 7 hours ago

tbykowsk commented 2 months ago

Hi, I followed ChatQnA Application readme and encountered some problems. I started with code from the master branch.

  1. In the step Start the Backend Service, any request to the streaming endpoint returns an error:
INFO:     Application startup complete.
    INFO:     Uvicorn running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
    INFO:     <IP>:50773 - "OPTIONS /v1/rag/chat_stream HTTP/1.1" 200 OK
    [rag - chat_stream] POST request: /v1/rag/chat_stream, params:{'query': 'What is the total revenue of Nike in 2023?"', 'knowledge_base_id': 'default'}
    [rag - chat_stream] history: []
    [rag - reload retriever] reload with index: rag-redis
    INFO:     <IP>:50773 - "POST /v1/rag/chat_stream HTTP/1.1" 200 OK
    score_threshold is deprecated. Use distance_threshold instead.score_threshold should only be used in similarity_search_with_relevance_scores.score_threshold will be removed in a future release.
    Metadata key source not found in metadata. Setting to None.
    Metadata fields defined for this instance: ['source', 'start_index']
    [...]
    Metadata key start_index not found in metadata. Setting to None.
    Metadata fields defined for this instance: ['source', 'start_index']
    ERROR:    Exception in ASGI application
    Traceback (most recent call last):
      File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 265, in __call__
        await wrap(partial(self.listen_for_disconnect, receive))
      File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap
        await func()
      File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 238, in listen_for_disconnect
        message = await receive()
                  ^^^^^^^^^^^^^^^
      File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 538, in receive
        await self.message_event.wait()
      File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
        await fut
    asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f0a23abf1d0

    During handling of the above exception, another exception occurred:

      + Exception Group Traceback (most recent call last):
      |   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
      |     result = await app(  # type: ignore[func-returns-value]
      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
      |     return await self.app(scope, receive, send)
      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |   File "/home/user/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
      |     await super().__call__(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
      |     raise exc
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
      |     await self.app(scope, receive, _send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
      |     await self.simple_response(scope, receive, send, request_headers=headers)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 148, in simple_response
      |     await self.app(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
      |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
      |     await route.handle(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
      |     await self.app(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
      |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
      |     raise exc
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
      |     await app(scope, receive, sender)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 75, in app
      |     await response(scope, receive, send)
      |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 258, in __call__
      |     async with anyio.create_task_group() as task_group:
      |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
      |     raise BaseExceptionGroup(
      | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
      +-+---------------- 1 ----------------
        | Traceback (most recent call last):
        |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 261, in wrap
        |     await func()
        |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 252, in stream_response
        |     chunk = chunk.encode(self.charset)
        |             ^^^^^^^^^^^^
        | AttributeError: 'NoneType' object has no attribute 'encode'
        +------------------------------------

This error causes that the Frontend Service is unresponsive, because it connects to /v1/rag/chat_stream endpoint which is broken.

When I reverted app/server.py to this commit the streaming endpoint started to work. It would be useful to add information to the instruction about a commit/release it was validated with.

  1. The Frontend Service allows to add new data sources Please upload your local file or paste a remote file link, and Chat will respond based on the content of the uploaded file. In the log new data sources are ingested correctly:
[rag - create] POST request: /v1/rag/create, filename:architecture-instruction-set-extensions-programming-reference.pdf
[rag - create kb folder] upload path: upload_dir/kb_0f19a9e4/upload_dir, persist path: upload_dir/kb_0f19a9e4/persist_dir
[rag - create] file saved to local path: upload_dir/kb_0f19a9e4/upload_dir/2024-04-17-16:54:11-architecture-instruction-set-extensions-programming-reference.pdf
[rag - create] starting to create local db...
[rag - create retriever] create with index: rag-rediskb_0f19a9e4
[nltk_data] Downloading package punkt to /home/user/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /home/user/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
`index_schema` does not match generated metadata schema.
If you meant to manually override the schema, please ignore this message.
index_schema: {'text': [{'name': 'content'}, {'name': 'source'}], 'numeric': [{'name': 'start_index'}], 'vector': [{'name': 'content_vector', 'algorithm': 'HNSW', 'datatype': 'FLOAT32', 'dims': 768, 'distance_metric': 'COSINE'}]}
generated_schema: {'text': [{'name': 'source'}], 'numeric': [{'name': 'start_index'}], 'tag': []}

[rag - create] kb created successfully
INFO:     <IP>:51563 - "POST /v1/rag/create HTTP/1.1" 200 OK
[rag - upload_link] POST request: /v1/rag/upload_link, link list:['https://en.wikipedia.org/wiki/American_white_pelican']
[rag - create kb folder] upload path: upload_dir/kb_2ce41686/upload_dir, persist path: upload_dir/kb_2ce41686/persist_dir
[rag - upload_link] starting to create local db...
start fetch %s... https://en.wikipedia.org/wiki/American_white_pelican
`index_schema` does not match generated metadata schema.
If you meant to manually override the schema, please ignore this message.
index_schema: {'text': [{'name': 'content'}, {'name': 'source'}], 'numeric': [{'name': 'start_index'}], 'vector': [{'name': 'content_vector', 'algorithm': 'HNSW', 'datatype': 'FLOAT32', 'dims': 768, 'distance_metric': 'COSINE'}]}
generated_schema: {'text': [{'name': 'source'}, {'name': 'identify_id'}], 'numeric': [], 'tag': []}

[rag - upload_link] kb created successfully
INFO:     <IP>:20643 - "POST /v1/rag/upload_link HTTP/1.1" 200 OK

However, new data sources do not seem to be instantly included during response generation. A restart of app/server.py is required for new information to be available, even though in the backend an index changes with each uploaded document, and is reloaded with each response, example of log: [rag - reload retriever] reload with index: rag-rediskb_147637c0. To confirm this problem, I have used a working app/server.py from the aforementioned commit, the rest of code was from the master branch.

Please look into those issues. Thanks!

lvliang-intel commented 2 months ago

Hi @tbykowsk,

Thanks for raising this issue. For the first issue, I fixed it by PR https://github.com/opea-project/GenAIExamples/pull/88. I'm currently attempting to reproduce the second issue. I'll keep you updated on my progress.

lvliang-intel commented 2 months ago

Hi @tbykowsk,

Currently, our knowledge base creation process lacks an appending strategy. Whenever a file is uploaded, a new knowledge base ID is generated. Similarly, when a html link is pasted, it also generates a new knowledge base ID. As a result, the frontend utilizes the most recent knowledge base ID as the default for chatting. Consequently, if a question pertains to the uploaded file, the system searches only within the knowledge base associated with the uploaded link.

To enhance this process, we plan to refine the code and implement a knowledge appending strategy. This strategy will involve appending new knowledge to existing knowledge bases rather than generating new IDs each time.

tbykowsk commented 2 months ago

Hi @lvliang-intel, thank you for such a quick response!

I can confirm that the first issue is resolved.

Thank you for the details about the knowledge base implementation. Maybe information how it functions could be added to Readme, so it is clear what to expect.

tbykowsk commented 1 month ago

@lvliang-intel, are you able to say when do you plan to implement a knowledge appending strategy?

apahleva commented 1 month ago

@lvliang-intel, I am following the steps in the README, and after I successfully build the Docker container and set up my Hugging Face token, I encounter an issue that I don't know how to resolve. Can you guide me?

curl -v --noproxy '*' 127.0.0.1:8080/generate -X POST -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":32}}' -H 'Content-Type: application/json' Note: Unnecessary use of -X or --request, POST is already inferred.

preethivenkatesh commented 2 weeks ago

@tbykowsk Please let us know if this issue is fixed and can be closed

letonghan commented 7 hours ago

@tbykowsk Since the dataprep microservice supports the knowledge base append strategy now, we will close this issue. Please let us know if there is other issues.