idiap / coqui-ai-TTS

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
https://coqui-tts.readthedocs.io
Mozilla Public License 2.0
565 stars 58 forks source link

[Bug] Zoo tests are flaky because of download errors #131

Open eginhard opened 1 week ago

eginhard commented 1 week ago

Describe the bug

The zoo tests regularly fail in CI because of temporary errors when downloading models, e.g. https://github.com/idiap/coqui-ai-TTS/actions/runs/11693792828/job/32566095448?pr=128

At leasts in tests it would be useful to retry the download 1-2 times, so one doesn't need to rerun the test manually

To Reproduce

Run tests in CI several times.

Expected behavior

Tests run without download errors.

Logs

ERROR:TTS.utils.manage:Bad zip file - https://github.com/coqui-ai/TTS/releases/download/v0.8.0_models/tts_models--sk--cv--vits.zip
Traceback (most recent call last):
  File "/home/runner/work/coqui-ai-TTS/coqui-ai-TTS/TTS/utils/manage.py", line 534, in _download_zip_file
    with zipfile.ZipFile(temp_zip_name) as z:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/zipfile.py", line 1313, in __init__
    self._RealGetContents()
  File "/home/runner/.local/share/uv/python/cpython-3.11.10-linux-x86_64-gnu/lib/python3.11/zipfile.py", line 1380, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
ERROR

Environment

TTS: 0.24.3
shavit commented 5 days ago

Yes, maybe rate limits cause timeouts. I'm not familiar with the uv cache option, but the downloads were not shared between runs in my tests.

eginhard commented 3 days ago

Good point, the downloads are not cached. I just tried that, but actually collecting all models reaches the storage limits of Github runners, so each model needs to be removed after use. To make caching work in this case likely requires some redesign of the test workflow. Retrying downloads could still be implemented.