cuducos / chunk

🧱 Chunk is a download manager for slow and unstable servers
MIT License
56 stars 3 forks source link

Uses HTTP Content-Range for downloading #21

Closed cuducos closed 1 year ago

cuducos commented 1 year ago

This PRs addresses #5 but not using parallel downloads. The files are sequentially downloaded chunk by chunk using the HTTP content-range header. Manually I tested this branch with:

$ go run main.go http://200.152.38.155/CNPJ/Paises.zip
2022/11/19 10:32:34 Downloaded http://200.152.38.155/CNPJ/Paises.zip (2745 bytes) to /var/folders/zb/0n9ntv254vq9zfhsvyq2g7680000gn/T/Paises.zip (2745 bytes).
2022/11/19 10:32:35 All download(s) finished successfully.
$ unzip /var/folders/zb/0n9ntv254vq9zfhsvyq2g7680000gn/T/Paises.zip 
Archive:  /var/folders/zb/0n9ntv254vq9zfhsvyq2g7680000gn/T/Paises.zip
  inflating: F.K03200$Z.D21008.PAISCSV  
$ head F.K03200\$Z.D21008.PAISCSV  && tail F.K03200\$Z.D21008.PAISCSV 
"000";"COLIS POSTAUX"
"013";"AFEGANISTAO"
"017";"ALBANIA"
"020";"ALBORAN-PEREJIL,ILHAS"
"023";"ALEMANHA"
"025";"ALEMANHA, REP. DEMOCRATICA DA"
"031";"BURKINA FASO"
"037";"ANDORRA"
"040";"ANGOLA"
"041";"ANGUILLA"
"866";"VIRGENS, ILHAS (EUA)"
"870";"FIJI"
"875";"WALLIS E FUTURNA, ILHAS"
"888";"ZAIRE"
"890";"ZAMBIA"
"895";"ZONA DO CANAL DO PANAMA"
"990";"PROV. DE NAVIOS E AERONAVES"
"997";"NAO DECLARADOS"
"998";"NAO DECLARADO PRELIMINAR"
"999";"NAO DECLARADOS"

I decided to change uint64 to int64 because I've found out that an error in conversion would make debugging way more difficult. Happy to revert that once the package is more stable.