search

mlcommons / training

Reference implementations of MLPerf™ training benchmarks
https://mlcommons.org/en/groups/training
Apache License 2.0
1.62k stars 560 forks source link

Problem downloading S3 bucket #746

Closed mahmoodn closed 5 months ago

mahmoodn commented 5 months ago

Hi, I followed the instructions in the large language model to download the S3 bucket, but apparently that doesn't work.

$ rclone config create mlc-training s3 provider=Cloudflare access_key_id=76ea42eadb867e854061a1806220ee1e secret_access_key=a53625c4d45e3ca8ac0df8a353ea3a41ffc3292aa25259addd8b7dc5a6ce2936 endpoint=https://c2686074cb2caf5cbaf6d134bdba8b47.r2.cloudflarestorage.com
[mlc-training]
type = s3
provider = Cloudflare
access_key_id = 76ea42eadb867e854061a1806220ee1e
secret_access_key = a53625c4d45e3ca8ac0df8a353ea3a41ffc3292aa25259addd8b7dc5a6ce2936
endpoint = https://c2686074cb2caf5cbaf6d134bdba8b47.r2.cloudflarestorage.com

$ rclone copy mlc-training:mlcommons-training-wg-public/gpt3/megatron-lm/dataset_c4_spm.tar ./ -P
2024/06/05 15:34:21 NOTICE: Time may be set wrong - time from "c2686074cb2caf5cbaf6d134bdba8b47.r2.cloudflarestorage.com" is 1h52m37.145836609s different from this computer
2024/06/05 15:34:21 ERROR : S3 bucket mlcommons-training-wg-public path gpt3/megatron-lm/dataset_c4_spm.tar: error reading source root directory: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : Attempt 1/3 failed with 1 errors and: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : S3 bucket mlcommons-training-wg-public path gpt3/megatron-lm/dataset_c4_spm.tar: error reading source root directory: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : Attempt 2/3 failed with 1 errors and: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : S3 bucket mlcommons-training-wg-public path gpt3/megatron-lm/dataset_c4_spm.tar: error reading source root directory: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
2024/06/05 15:34:21 ERROR : Attempt 3/3 failed with 1 errors and: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         1.3s
2024/06/05 15:34:21 Failed to copy: RequestTimeTooSkewed: The difference between the request time and the server's time is too large.
        status code: 403, request id: , host id:

I used the same rclone config command as the readme file says. I am not sure if I have to do anything more or not. Any thoughts on that?