vertexai.preview.rag.list_corpora() raises error

[kunj-gandhi12]

Hi,

1. From past weeks, I am working on LlamaIndex on Vertex AI using RAG.
2. Everything was working fine. But today when i ran code, that involves vertexai.preview.rag.list_corpora() to list all the corpus it gives below error.
3. Also, isn't there any better way to provide corpus name. Writing entire /projects/project_id/locations/location_id/ragCorpora/rag_id is no easy, instead it should be just corpus display name that we use to create corpus.

 File "/home/user/userprojects/.venv/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 1181, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "Internal error encountered."
        debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2404:6800:4009:80d::200a%5D:443 {created_time:"2024-08-06T19:27:57.847920505+05:30", grpc_status:13, grpc_message:"Internal error encountered."}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/vertexai/preview/rag/rag_data.py", line 164, in list_corpora
    pager = client.list_rag_corpora(request=request)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/google/cloud/aiplatform_v1beta1/services/vertex_rag_data_service/client.py", line 1105, in list_rag_corpora
    response = rpc(
               ^^^^
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InternalServerError: 500 Internal error encountered.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/userprojects/sample.py", line 11, in <module>
    corpora = rag.list_corpora()
              ^^^^^^^^^^^^^^^^^^
  File "/home/user/userprojects/.venv/lib/python3.12/site-packages/vertexai/preview/rag/rag_data.py", line 166, in list_corpora
    raise RuntimeError("Failed in listing the RagCorpora due to: ", e) from e
RuntimeError: ('Failed in listing the RagCorpora due to: ', InternalServerError('Internal error encountered.'))

Due to this existing code has stopped running. What can be the reasons. How to solve this.

Also, i want to ask, is there any way to see where this rag data, metadata is stored in GCP VertexAI. or the Only way to communicate is via SDK and API?

[nlorenzen1]

This is also affecting rag.get_corpus()

[qizzzh]

Sorry for the interruption! We've addressed the issues and the APIs should be working again. Please let me know if the problem persists.

[kunj-gandhi12]

Ya, its working. Thanks!!

Also, i want to ask like is it possible to view this rag service via google cloud console, or its only accesible via SDK.

[qizzzh]

We are trying to get a UI for it. Stay tuned!

[kunj-gandhi12]

Ok, thanks for the help. !!