search

NVIDIA / GenerativeAIExamples

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
Apache License 2.0
2.47k stars 535 forks source link

"Failed to get response from /generate endpoint of chain-server. Check if the fastapi server in chain-server is up. Refer to chain-server logs for details." #232

Open wyw1267 opened 3 weeks ago

wyw1267 commented 3 weeks ago

When I run the langchain example, the docker command can run successfully. But when I use the brower to chat. It will report "Failed to get response from /generate endpoint of chain-server. Check if the fastapi server in chain-server is up. Refer to chain-server logs for details.".

MarkoBrie commented 2 weeks ago

I get the same message for the GenerativeAIExamples/RAG/examples/basic_rag/llamaindex. Message: "Failed to get response from /generate endpoint of chain-server. Check if the fastapi server in chain-server is up. Refer to chain-server logs for details."

MarkoBrie commented 2 weeks ago

I get the same message for the GenerativeAIExamples/RAG/examples/basic_rag/llamaindex. Message: "Failed to get response from /generate endpoint of chain-server. Check if the fastapi server in chain-server is up. Refer to chain-server logs for details."

Somehow the export of the NVIDIA_API_KEY was ignored. I repeated the export and the error was not shown anymore.

thalapandi commented 2 weeks ago

https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints/

lbliii commented 2 weeks ago

There seems to be a missing dependency issue in the RAG/src/chain-server/tracing.py file at the following location:

from RAG.tools.observability.langchain import opentelemetry_callback as langchain_otel_cb
from RAG.tools.observability.llamaindex import opentelemetry_callback as llama_index_otel_cb

When I inspect the exited chain-server container, I see the following error:

from RAG.tools.observability.langchain import opentelemetry_callback as langchain_otel_cb
2024-11-09 11:38:58 ModuleNotFoundError: No module named 'RAG.tools.observability'