TABLES_ROOT may be an S3 URI, e.g. s3://my-bucket-name/dst/dir.
The cache/info endpoint will read the remote s3 cache_info.json file if TABLES_ROOT is a URL.
The cache.zip endpoint will return an HTTP redirect if the TABLES_ROOT location is not local, however the API doesn't support this as the URL parameters are no longer valid for external URLs.
The cache generation now has a compress option which is required to save a zip file. If set, the tables are compressed in memory and written to a zip file.
When compress flag is set the tables are written to a temp directory before compression.
generate_cache now supports an S3 TABLES_ROOT URI. Tables are directly written to S3 using the pyarrow FileSystem API, which does not require extra dependencies.
s3://my-bucket-name/dst/dir.