[Question]: ERROR from llama-deploy httpx.ConnectError: All connection attempts failed

[BlueKiji77]

Question Validation

• [X] I have searched both the documentation and discord for an answer.

Question

I am working in a cloud environment; lightning ai studios and trying to launch a simple app on it. Following is my main.py code

from src import Config
from llama_deploy import deploy_workflow, WorkflowServiceConfig, ControlPlaneConfig
from src import ReActAgent
from src.utils import download_10k_reports
from src.utils import load_hf_model
from src.utils import initialize_groq_model
from src.ingest import setup_indices
from llama_index.core import PromptTemplate
from src.query_engine import HybridRetriver, StuffedContextQueryEngine
from llama_index.core.postprocessor import SimilarityPostprocessor
from llama_index.postprocessor.colbert_rerank import ColbertRerank
from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.core.indices.vector_store.retrievers import VectorIndexRetriever

async def main(local_model=False):
    download_10k_reports()
    config = Config()
    if local_model:
        hf_models = load_hf_model(config.hf_tiny_model, config.hf_embed_model)
    else:
        hf_models = load_hf_model(config.hf_tiny_model, config.hf_embed_model, embed_model_only=True)
        groq_llama_8b = initialize_groq_model("llama3-8b-8192")
        groq_llama_70b = initialize_groq_model("llama3-70b-8192")

    indices = setup_indices(hf_models['embed_model'])
    lyft_index = indices['lyft_index']
    uber_index = indices['uber_index']

    # lyft_retriever = HybridRetriver(index=lyft_index, vector_similarity_top_k=20, bm25_similarity_top_k=20, fusion_similarity_top_k=20, llm=groq_llama_8b)
    # uber_retriever = HybridRetriver(index=uber_index, vector_similarity_top_k=20, bm25_similarity_top_k=20, fusion_similarity_top_k=20, llm=groq_llama_8b)

    lyft_retriever = VectorIndexRetriever(index=lyft_index, similarity_top_k=20)
    uber_retriever = VectorIndexRetriever(index=uber_index, similarity_top_k=20)

    colbert_reranker = ColbertRerank(
        top_n=10,
        model="colbert-ir/colbertv2.0",
        tokenizer="colbert-ir/colbertv2.0",
        keep_retrieval_score=True,
    )

    postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.7),
                    colbert_reranker]                       

    qa_prompt = PromptTemplate(
        """\
    Context information is below.
    ---------------------
    {context_str}
    ---------------------
    Given the context information and not prior knowledge, answer the query.
    Query: {query_str}
    Answer: \
    """
    )

    lyft_query_engine = StuffedContextQueryEngine(
                        retriever=lyft_retriever, 
                        qa_prompt=qa_prompt,
                        llm=groq_llama_8b,
                        node_postprocessors=postprocessors
                        )

    uber_query_engine = StuffedContextQueryEngine(
                        retriever=uber_retriever, 
                        qa_prompt=qa_prompt,
                        llm=groq_llama_8b,
                        node_postprocessors=postprocessors
                        )

    query_engine_tools = [
        QueryEngineTool(
            query_engine=lyft_query_engine,
            metadata=ToolMetadata(
                name="lyft_10k",
                description="Provides information about Lyft financials for year 2021",
            ),
        ),
        QueryEngineTool(
            query_engine=uber_query_engine,
            metadata=ToolMetadata(
                name="uber_10k",
                description="Provides information about Uber financials for year 2021",
            ),
        ),
    ]
    await deploy_workflow(
        ReActAgent(
            llm=groq_llama_8b,
            tools=query_engine_tools,
            timeout=120,
            verbose=False
        ),
        WorkflowServiceConfig(host="127.0.0.1", port=8000, service_name="my_workflow"),
        ControlPlaneConfig(host="127.0.0.1", port=8000),
    )

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

ERROR

Traceback (most recent call last):
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
    yield
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_transports/default.py", line 366, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
    raise exc
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
    stream = await self._connect(request)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_async/connection.py", line 122, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
    return await self._backend.connect_tcp(
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp
    with map_exceptions(exc_map):
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: All connection attempts failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/teamspace/studios/this_studio/react_agent/src/main.py", line 100, in <module>
    asyncio.run(main())
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/teamspace/studios/this_studio/react_agent/src/main.py", line 87, in main
    await deploy_workflow(
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/llama_deploy/deploy/deploy.py", line 198, in deploy_workflow
    response = await client.get(f"{control_plane_url}/queue_config")
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_client.py", line 1757, in get
    return await self.request(
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_client.py", line 1530, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_client.py", line 1719, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_transports/default.py", line 365, in handle_async_request
    with map_httpcore_exceptions():
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed

Does anyone have any idea as to the possible source of this error?

[dosubot[bot]]

The httpx.ConnectError: All connection attempts failed error typically indicates a problem with connecting to the specified host or port. Here are some steps to troubleshoot and resolve the issue:

1. Check Host and Port: Ensure that the host (127.0.0.1) and port (8000) specified in your WorkflowServiceConfig and ControlPlaneConfig are correct and that the service is running on that address.

2. Service Availability: Verify that the service you are trying to connect to is up and running. You can use tools like curl or telnet to check if the port is open and accepting connections.

3. Firewall and Network Settings: Check if there are any firewall rules or network settings that might be blocking the connection to the specified host and port.

4. Dependencies and Environment: Ensure all necessary dependencies are installed and that your environment is correctly set up. This includes verifying that the llama_index and related packages are properly installed.

5. Logs and Debugging: Check any available logs for more detailed error messages that might provide additional context about the connection failure.

These steps should help you identify and resolve the connection issue ^[1].

To continue talking to Dosu, mention @dosu.

[logan-markewich]

@BlueKiji77 fyi, we have an actual llama-deploy repo for llama-deploy issues. Moving this over there

Normally, I think this issue would occur because you didn't run deploy_core()

You can see from the error, its failing to on response = await client.get(f"{control_plane_url}/queue_config")

[BlueKiji77]

@BlueKiji77 fyi, we have an actual llama-deploy repo for llama-deploy issues. Moving this over there

Normally, I think this issue would occur because you didn't run deploy_core()

You can see from the error, its failing to on response = await client.get(f"{control_plane_url}/queue_config")

Thank you very much Logan, it works now. Have a Great day