Get Object mainly makes 2 update to spped up get object:
remove unnecessary head object
add sp endpoint as an option for get object, which can skip refresh SPs and get SP host steps.
Batch Upload update:
Supports batch file upload
Rationale
Get Object:
To make the user have better download experience via gnfd-cmd.
For 50K file download, the speed on normal computer change from ~4s to ~2s (1.6s for files in AP region and 2.4s for files in US region), speed up around 100%. If use EC2 machine, download 50K file in same region can speed up to ~0.8s.
For large files e.g. 500MB download, the download speed also decrease ~2s. Since we optimize the steps before download but not download it self.
Batch Upload:
Recursively uploading files requires management/viewing progress, as well as retry capabilities
Example
Get Object:
./gnfd-cmd --config config.toml --passwordfile password.txt object get --spEndpoint https://gnfd-testnet-sp3.nodereal.io gnfd://annie-ap-sp3/flower1.jpeg
Description
Get Object mainly makes 2 update to spped up get object:
Batch Upload update: Supports batch file upload
Rationale
Get Object: To make the user have better download experience via gnfd-cmd.
For 50K file download, the speed on normal computer change from ~4s to ~2s (1.6s for files in AP region and 2.4s for files in US region), speed up around 100%. If use EC2 machine, download 50K file in same region can speed up to ~0.8s.
For large files e.g. 500MB download, the download speed also decrease ~2s. Since we optimize the steps before download but not download it self.
Batch Upload: Recursively uploading files requires management/viewing progress, as well as retry capabilities
Example
Get Object: ./gnfd-cmd --config config.toml --passwordfile password.txt object get --spEndpoint https://gnfd-testnet-sp3.nodereal.io gnfd://annie-ap-sp3/flower1.jpeg
Batch Upload: ./gnfd-cmd --home ./ bucket create gnfd://constwz5 ./gnfd-cmd --home ./ object put --recursive ~/Desktop/t-folder gnfd://constwz5 .gnfd-cmd --home ./ task status --taskId 193977d7-93b1-48d3-ad83-cf2faf9feeb2 ./gnfd-cmd --home ./ task retry --taskId 735b85cd-251c-4f73-a81e-7c60da2bf3fd ./gnfd-cmd --home ./ task delete --taskId 034b5786-bb62-4620-a145-c2de9a1e70f6
Changes
Notable changes: