search

googleapis / google-cloud-python

Google Cloud Client Library for Python
https://googleapis.github.io/google-cloud-python/
Apache License 2.0
4.81k stars 1.52k forks source link

429 received metadata size exceeds hard limit #13119

Open mcantrell opened 4 days ago

mcantrell commented 4 days ago

Determine this is the right repository

Summary of the issue

Context

Using the TextToSpeechLongAudioSynthesizeAsyncClient to produce a Neural2 voice file results in an gRPC error.

Expected Behavior: The speech file to successfully land in the provided GCP storage bucket.

Actual Behavior: The client produces the following error:

 google.api_core.exceptions.ResourceExhausted: 429 received metadata size exceeds hard limit (18013 vs. 16384);  grpc-status:44B grpc-message:17969B

API client name and version

google-cloud-texttospeech 2.17.2

Reproduction steps: code

import os

import pytest
from google.api_core.client_options import ClientOptions
from google.cloud import texttospeech
from google.protobuf.message import Message

@pytest.mark.asyncio
async def test_metadata_error(pytestconfig, config):
    ssml = read_test_document(pytestconfig, "tests/resources/problems/tts-metadata-error.txt")
    client = texttospeech.TextToSpeechLongAudioSynthesizeAsyncClient(
        client_options=ClientOptions(quota_project_id=config.gcp_project_id)
    )
    # noinspection PyTypeChecker
    tts_input = texttospeech.SynthesisInput(ssml=ssml)
    # noinspection PyTypeChecker
    audio_config = texttospeech.AudioConfig(
        audio_encoding=texttospeech.AudioEncoding.LINEAR16
    )
    # noinspection PyTypeChecker
    voice = texttospeech.VoiceSelectionParams(
        language_code="en-GB",
        name="en-GB-Neural2-F"
    )
    # noinspection PyTypeChecker
    request = texttospeech.SynthesizeLongAudioRequest(
        parent=f"projects/{config.gcp_project_id}/locations/us-central1",
        input=tts_input,
        audio_config=audio_config,
        voice=voice,
        output_gcs_uri=f"gs://{config.gcs_internal_bucket_name}/test/metadata_error.wav",
    )
    operation = await client.synthesize_long_audio(request=request)
    result: Message = await operation.result(timeout=300)
    assert result

def read_test_document(pytestconfig, relative_path: str) -> str:
    file_path = os.path.join(pytestconfig.rootpath, relative_path) 
    with open(file_path, "r") as file:
        return file.read()

Reproduction steps: supporting files

tts-metadata-error.txt

Reproduction steps: actual results

 google.api_core.exceptions.ResourceExhausted: 429 received metadata size exceeds hard limit (18013 vs. 16384);  grpc-status:44B grpc-message:17969B

Reproduction steps: expected results

the test should not error

OS & version + platform

MacOS 14.5

Python environment

Python 3.11.9

Python dependencies

Package                           Version     Editable project location
--------------------------------- ----------- ------------------------------------------------------
aiohappyeyeballs                  2.4.0
aiohttp                           3.10.5
aiosignal                         1.3.1
annotated-types                   0.7.0
anthropic                         0.34.2
anyio                             4.5.0
attrs                             24.2.0
cachetools                        5.5.0
cartesia                          1.0.13
certifi                           2024.8.30
cffi                              1.17.1
charset-normalizer                3.3.2
click                             8.1.7
courts-db                         0.10.25
cryptography                      43.0.1
dataclasses-json                  0.6.7
decorator                         5.1.1
defusedxml                        0.7.1
Deprecated                        1.2.14
deprecation                       2.1.0
distro                            1.9.0
dnspython                         2.6.1
email_validator                   2.2.0
eyecite                           2.6.4
fast-diff-match-patch             2.1.0
fastapi                           0.111.1
fastapi-cli                       0.0.5
filelock                          3.16.0
frozenlist                        1.4.1
fsspec                            2024.9.0
google-api-core                   2.19.2
google-auth                       2.34.0
google-cloud-appengine-logging    1.4.5
google-cloud-audit-log            0.3.0
google-cloud-bigquery             3.25.0
google-cloud-core                 2.4.1
google-cloud-logging              3.11.2
google-cloud-recaptcha-enterprise 1.22.1
google-cloud-secret-manager       2.20.2
google-cloud-storage              2.18.2
google-cloud-texttospeech         2.17.2
google-cloud-vision               3.7.4
google-crc32c                     1.6.0
google-resumable-media            2.7.2
googleapis-common-protos          1.65.0
greenlet                          3.1.0
grpc-google-iam-v1                0.13.1
grpcio                            1.66.1
grpcio-status                     1.62.3
h11                               0.14.0
httpcore                          1.0.5
httptools                         0.6.1
httpx                             0.27.2
huggingface-hub                   0.24.7
idna                              3.8
importlib_metadata                8.4.0
importlib_resources               6.4.5
inflection                        0.5.1
iniconfig                         2.0.0
iterators                         0.2.0
Jinja2                            3.1.4
jiter                             0.5.0
json_repair                       0.25.3
jsonpatch                         1.33
jsonpointer                       3.0.0
lancedb                           0.13.0
langchain                         0.2.16
langchain-anthropic               0.1.23
langchain-community               0.2.16
langchain-core                    0.2.39
langchain-experimental            0.0.65
langchain-openai                  0.1.23
langchain-text-splitters          0.2.4
langsmith                         0.1.118
limits                            3.13.0
lxml                              5.3.0
markdown-it-py                    3.0.0
MarkupSafe                        2.1.5
marshmallow                       3.22.0
mdurl                             0.1.2
multidict                         6.1.0
mypy-extensions                   1.0.0
neo4j                             5.24.0
numpy                             1.26.4
openai                            1.44.1
opentelemetry-api                 1.27.0
orjson                            3.10.7
overrides                         7.7.0
packaging                         24.1
pandas                            2.2.2
pip                               24.1
pluggy                            1.5.0
proto-plus                        1.24.0
protobuf                          4.25.4
py                                1.11.0
pyahocorasick                     2.1.0
pyarrow                           17.0.0
pyasn1                            0.6.1
pyasn1_modules                    0.4.1
pycparser                         2.22
pydantic                          2.9.1
pydantic_core                     2.23.3
Pygments                          2.18.0
PyJWT                             2.9.0
pylance                           0.17.0
PyMuPDF                           1.24.10
PyMuPDFb                          1.24.10
pytest                            8.3.3
pytest-asyncio                    0.23.8
python-dateutil                   2.9.0.post0
python-dotenv                     1.0.1
python-magic                      0.4.27
python-multipart                  0.0.9
pytz                              2024.2
PyYAML                            6.0.2
regex                             2024.9.11
reporters-db                      3.2.44
requests                          2.32.3
retry                             0.9.2
rich                              13.8.1
rsa                               4.9
shellingham                       1.5.4
six                               1.16.0
sniffio                           1.3.1
SQLAlchemy                        2.0.34
starlette                         0.37.2
tavily-python                     0.3.9
tenacity                          8.5.0
tiktoken                          0.7.0
tokenizers                        0.20.0
tqdm                              4.66.5
typer                             0.12.5
typing_extensions                 4.12.2
typing-inspect                    0.9.0
tzdata                            2024.1
urllib3                           2.2.3
uvicorn                           0.30.6
uvloop                            0.20.0
watchfiles                        0.24.0
websockets                        13.0.1
wrapt                             1.16.0
yarl                              1.11.1
zipp                              3.20.1
ohmayr commented 2 days ago

@mcantrell Thanks for reporting this. The error seems to be emitted from the gRPC layer. I've filed https://github.com/grpc/grpc/issues/37852 which you can follow for any status updates. We will work with the folks there and investigate what the root cause is.