Rewrite client-py's upload functions to use tus.io-based API

lablup / backend.ai

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, TPU, IPU and other NPUs.

https://www.backend.ai

GNU Lesser General Public License v3.0

517 stars 153 forks source link

Rewrite client-py's upload functions to use tus.io-based API #182

Open achimnol opened 4 years ago

achimnol commented 4 years ago

Like javascript-based clients, let's use the tus.io protocol for the client-py. It has the following advantages:

All the advantages of tus.io protocol such as resuming uploads
More accurate progress bars
Authentication is done first before actually sending the payload, reducing possible network traffic waste upon authentication failure in the legacy API

achimnol commented 4 years ago

There is tus-py-client and it seems to support aiohttp-based async uploader though async uploader is not documented & not released yet.