search

prasmussen / gdrive

Google Drive CLI Client
MIT License
9k stars 1.19k forks source link

Split upload (stdin) to multiple files #208

Open ghfields opened 7 years ago

ghfields commented 7 years ago

I have a very large stdin that I would like to feed gdrive and have it upload it as split files.

Also the reverse would be desired: a way to download the split files and pass it as a stdin of another program.

ghfields commented 7 years ago

I got a temporary workaround.

zfs send -R pool/ROOT/ubuntu@1609201008 |parallel -u -P 3 --pipe --recend '' --block 1G 'gdrive-linux-x64 upload - ubuntu,1609201008-{#}.zfssplit'

This splits the stdin into 1GB chunks and passes it to up to 3 cocurrent gdrive instances. It in my case, 39 uploads later, I have a successful upload. It still would be nice if gdrive natively split the pipe.

I also have to work on finding a way to 'cat' the split downloads directly into a pipe to be fed into "zfs receive".