Closed guozhiyao closed 1 month ago
I try to download from the url, such as wget -P ./ -c https://data.together.xyz/redpajama-data-1T/v1.0.0/wikipedia/wiki.jsonl. But it meets error as below
wget -P ./ -c https://data.together.xyz/redpajama-data-1T/v1.0.0/wikipedia/wiki.jsonl
63888150K .......... .......... .......... .......... .......... 54% 26.5M 82m10s 63888200K .......... .......... .......... .......... .......... 54% 28.9M 82m10s 63888250K .......... .......... .......... .......... .......... 54% 27.6M 82m10s 63888300K .......... .......... .......... .......... 54% 26.1M=98m14s 2023-06-08 12:02:19 (10.6 MB/s) - Connection closed at byte 65421660160. Retrying. --2023-06-08 12:02:20-- (try: 2) https://data.together.xyz/redpajama-data-1T/v1.0.0/wikipedia/wiki.jsonl Connecting to data.together.xyz (data.together.xyz)|104.26.15.50|:443... connected. HTTP request sent, awaiting response... 206 Partial Content Length: 120142320713 (112G), 54720660553 (51G) remaining Saving to: ‘./wiki.jsonl’ [ skipping 63888300K ] 63888300K ,,,,,,,,,, ,,,,,,,,,, ,,,,,,,,,, ,,,,,,,,,, .......... 54% 7.02M 10h19m 63888350K .......... .......... .......... .......... .......... 54% 938K 14h54m 63888400K .......... .......... .......... .......... .......... 54% 6.70M 9h6m 63888450K .......... .......... .......... .......... .......... 54% 11.6M 6h39m 63888500K .......... .......... .......... .......... .......... 54% 10.2M 5h24m 63888550K .......... .......... .......... .......... .......... 54% 215K 17h39m 63888600K .......... .......... .......... .......... .......... 54% 992K 17h13m 63888650K .......... .......... .......... .......... .......... 54% 496K 18h59m 63888700K .......... .......... .......... .......... .......... 54% 659K 19h25m 63888750K .......... .......... .......... .......... .......... 54% 1.44M 18h24m 63888800K .......... .......... .......... .......... .......... 54% 1017K 18h1m 63888850K .......... .......... .......... .......... .......... 54% 1.27M 17h26m 63888900K .......... .......... .......... .......... .......... 54% 1.30M 16h55m 63888950K .......... .......... .......... .......... .......... 54% 1.21M 16h33m 63889000K .......... .......... .......... .......... .......... 54% 1.07M 16h20m 63889050K .......... .......... .......... .......... .......... 54% 1.55M 15h53m 63889100K .......... .......... .......... .......... .......... 54% 1.97M 15h21m 63889150K .......... .......... .......... .......... .......... 54% 1.76M 14h56m 63889200K .......... .......... .......... .......... .......... 54% 1.52M 14h38m 63889250K .......... .......... .......... .......... .......... 54% 2.09M 14h14m 63889300K .......... .......... .......... .......... .......... 54% 2.22M 13h51m 63889350K .......... 54% 1.21M=1.0s Cannot write to ‘./wiki.jsonl’ (Success).
How to fix it?
Hi @guozhiyao , the reason for this is most likely that you ran out of disk space (the wikipedia slice requires ~112G of space).
I try to download from the url, such as
wget -P ./ -c https://data.together.xyz/redpajama-data-1T/v1.0.0/wikipedia/wiki.jsonl
. But it meets error as belowHow to fix it?