ajschmidt8
commented
1 year ago To provide additional context...
data.rapids.ai serves the contents of the rapidsai-data S3 bucket via an AWS CloudFront distribution.
The benefits of using a CloudFront distribution are:
- dataset files are served from an endpoint closest to individual end-users. Therefore latency is decreased, which makes datasets download faster
- data transfer costs out of AWS are cheaper
Therefore, it's in everyone's best interest to start using the new data.rapids.ai URLs for downloading datasets.
At some point in the future, the S3 URLs will be disabled and datasets will only be retrievable from data.rapids.ai.
Update datasets download URL to reduce latency and costs