lhotse-speech / lhotse

Tools for handling speech data in machine learning projects.
https://lhotse.readthedocs.io/en/latest/
Apache License 2.0
902 stars 204 forks source link

Support for reading data from AIStore using Python SDK #1354

Closed pzelasko closed 3 weeks ago

pzelasko commented 3 weeks ago

Support for AIStore dataloading. To enable, pip install aistore and set env var AIS_ENDPOINT=http://address:port.

It supports reading data from URLs such as ais://bucket-name/path/to/object.ext. When AIS_ENDPOINT is defined, we'll also redirect links such as s3://bucket/object.ext (and other object stores supported by AIStore) to AIStore rather than use smart_open.